[go-nuts] Re: are Go generics more like C++ or more like Java?

[Ugorji Nwoke]

To your direct questions, the answer is a mix of both: C++ style where the 
generic block is a template and code is essentially generated for each type 
known at compile time with static dispatch, and java-like where it is 
similar to an interface call with dynamic dispatch. The compiler will 
strike the right balance of binary size, compile time and execution speed. 
To follow its heuristics rules (which will likely not be formally published 
to prevent people from depending on it), you might have to stay close to 
the development for that.

On Friday, February 25, 2022 at 5:15:47 AM UTC-5 markus@gain.pro wrote:

> Correction:
>
> - it must be possible to express that T is comparable (so that you can use 
> map[K]V, were K must be comparable)
> - it must be possible restrict T to a concrete list of types (f.e. min(a, 
> b T) T must be possible to write)
>
> On Friday, February 25, 2022 at 11:13:01 AM UTC+1 Markus Heukelom wrote:
>
>> I think the consensus in the Go community is to refrain from comparing Go 
>> language features with other programming languages. Rationale ~:
>>
>> - it is highly contentious
>> - it is very difficult to answer, it's like asking "is purple more blue 
>> or more red"
>> - no matter the answer, it will not help you a lot to write better 
>> programs or be more productive in Go
>>
>> However, maybe your real question is more like "what are/were the guiding 
>> principals for generics in Go?".   As far as I understand, generics in Go 
>> follow the same principles as other language features in Go:
>>
>> - compilation should be very fast, but trade-offs are allowed so it does 
>> not need to be maximally fast
>> - language features should be as orthogonal as possible, but trade-offs 
>> are allowed so they don't need to be maximally orthogonal
>> - it should be a simple as possible but no simpler
>> - ... 
>>
>> For generics this has resulted in:
>>
>> - exotic use-cases are not supported (for example having an integer 
>> *constant* as *generic parameter*, such as you see in C++ fixed size matrix 
>> templates, is not supported)
>> - it must be possible to express that T is comparable (f.e. min(a, b T) T 
>> must be possible to write)
>> - it must be possible restrict T to a concrete list of types 
>> - it must be possible to restrict T to a type with method set that equals 
>> that of some interface (I *personally* think this violates the principle of 
>> orthogonality too much and will lead to confusion and design debate)  
>>
>> Of course, neither list is exhaustive and is only my personal 
>> understanding of things.
>>
>> -Markus
>>
>> On Wednesday, February 23, 2022 at 9:41:14 PM UTC+1 Jason E. Aten wrote:
>>
>>> Back in 2009, Russ wrote a blog on generics, talking about the tradeoffs 
>>> in providing generics:
>>>
>>> "The generic dilemma is this: *do you want slow programmers, slow 
>>> compilers and bloated binaries, or slow execution times? " -- *
>>> https://research.swtch.com/generic
>>>
>>> I haven't had the bandwidth to follow the generics detailed 
>>> discussions.  I was just curious, at a high level, for creating a mental 
>>> model of Go's generics: which approach was taken?
>>>
>>> Thanks!
>>>
>>> J
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8edf65b3-f499-4446-908e-8e35d3070739n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

['Ugorji Nwoke' via golang-nuts]

Than,

Appreciate you saying that. And you're very welcome.

On Tue, Mar 30, 2021 at 8:29 PM Than McIntosh  wrote:

> Thank you @Ugorji Nwoke   , I appreciate your going
> the extra mile to include gollvm in your testing.
>
> Cheers, Than
>
>
> On Tue, Mar 30, 2021 at 11:17 AM Ugorji Nwoke  wrote:
>
>> Ugorji here - author of the github.com/ugorji/go/codec
>> <https://pkg.go.dev/github.com/ugorji/go/codec> package.
>>
>> The notes below are for folks that are interested in why we use unsafe,
>> and how we mitigate concerns around it.
>>
>> As Peter mentioned above, you can pass the build tag "codec.safe" to
>> bypass using unsafe code where we try to reach deeper into the runtime
>> implementation to optimize.
>>
>> I have been supporting gccgo for a while now, but never tested with
>> gollvm, as it has not been released at this time. I built a gollvm version
>> yesterday (from source) and tested it, and made some changes (see here
>> <https://github.com/ugorji/go/commit/1768cf4d4babceec77d8ab0da478c58c9cb9aa99>
>> and here
>> <https://github.com/ugorji/go/commit/12f7e67b2326c6cc05bd3fea410164a18108d6a3>)
>> so that gollvm will work in the default (high-performance) mode. I plan to
>> cut a new release v1.2.5 sometime this week with those changes.
>>
>> For those who care about why we support unsafe and dig into the runtime
>> internals, please read below.
>>
>> To illustrate the benefit, look at unsafe vs codec.safe benchmark results
>> below:
>> ```
>>  tags: "" (default high-performance mode using unsafe carefully) 
>> Benchmark__Json___Encode-86921 152808 ns/op   24 B/op
>>  1 allocs/op
>> Benchmark__Json___Encode-85622 197863 ns/op10048 B/op
>>381 allocs/op
>>
>>  tags: "codec.safe" 
>> Benchmark__Json___Decode-82587 415595 ns/op71878 B/op
>>592 allocs/op
>> Benchmark__Json___Decode-82167 478122 ns/op96812 B/op
>>   1456 allocs/op
>>
>>  tags: x generated 
>> Benchmark__Json___Encode-88694 120519 ns/op 1528 B/op
>>  6 allocs/op
>> Benchmark__Json___Decode-82960 349320 ns/op70469 B/op
>>589 allocs/op
>> ```
>>
>> This benchmarks show that using unsafe carefully can cut down allocations
>> dramatically. Encoding allocation goes from 381 to 1, while decoding goes
>> from 1456 to 592. Those numbers using unsafe rival the allocation numbers
>> that we get using code-generation (as seen above), and the run time starts
>> to trend within 25% of the code-generation numbers, and 25% better than the
>> non-unsafe run time.
>>
>> We limited the surface area that is exposed to unsafe (basically 1 file,
>> helper_unsafe.go, with some variant for gccgo/gollvm where some linkname
>> references do not exist or work differently), so we can quickly edit the
>> code and know where bugs are coming from. Many other packages that try to
>> optimize json/cbor/msgpack encoding and decoding use some variation of the
>> same ideas here.
>>
>> I test the code  for the last 5 go releases on each github checkin, via
>> travis CI. I also run with the standard compiler and gccgo locally before
>> cutting a release. I have now added gollvm to my pre-release validation
>> script, so it is validated before I cut the release. Caveat: I test with
>> the installed versions of gccgo from ubuntu, and locally built gollvm.
>> Since gollvm is not released yet, the version I test with may be old
>> (building gollvm takes roughly 1 hour on my computer).
>>
>> If you see any further issues, please file a bug and I will jump on it:
>> https://github.com/ugorji/go/issues/new
>>
>> Thanks.
>> On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:
>>
>>> You haven't said whether you followed the "safe" instructions for
>>> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go,
>>> which uses go:linkname.
>>>
>>> Package Documentation for github.com/ugorji/go/codec
>>> https://github.com/ugorji/go/blob/master/codec/README.md
>>>
>>> This package will carefully use 'package unsafe' for performance reasons
>>> in specific places. You can build without unsafe use by passing the safe or
>>> appengine tag i.e. 'go install -tags=codec.safe
>>>
>>> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>>>
>>> go test

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[Ugorji Nwoke]

Ugorji here - author of the github.com/ugorji/go/codec 
 package.

The notes below are for folks that are interested in why we use unsafe, and 
how we mitigate concerns around it.

As Peter mentioned above, you can pass the build tag "codec.safe" to bypass 
using unsafe code where we try to reach deeper into the runtime 
implementation to optimize.

I have been supporting gccgo for a while now, but never tested with gollvm, 
as it has not been released at this time. I built a gollvm version 
yesterday (from source) and tested it, and made some changes (see here 
 
and here 
) 
so that gollvm will work in the default (high-performance) mode. I plan to 
cut a new release v1.2.5 sometime this week with those changes.

For those who care about why we support unsafe and dig into the runtime 
internals, please read below.

To illustrate the benefit, look at unsafe vs codec.safe benchmark results 
below:
```
 tags: "" (default high-performance mode using unsafe carefully) 
Benchmark__Json___Encode-86921 152808 ns/op   24 B/op   
 1 allocs/op
Benchmark__Json___Encode-85622 197863 ns/op10048 B/op   
   381 allocs/op

 tags: "codec.safe" 
Benchmark__Json___Decode-82587 415595 ns/op71878 B/op   
   592 allocs/op
Benchmark__Json___Decode-82167 478122 ns/op96812 B/op   
  1456 allocs/op

 tags: x generated 
Benchmark__Json___Encode-88694 120519 ns/op 1528 B/op   
 6 allocs/op
Benchmark__Json___Decode-82960 349320 ns/op70469 B/op   
   589 allocs/op
```

This benchmarks show that using unsafe carefully can cut down allocations 
dramatically. Encoding allocation goes from 381 to 1, while decoding goes 
from 1456 to 592. Those numbers using unsafe rival the allocation numbers 
that we get using code-generation (as seen above), and the run time starts 
to trend within 25% of the code-generation numbers, and 25% better than the 
non-unsafe run time.

We limited the surface area that is exposed to unsafe (basically 1 file, 
helper_unsafe.go, with some variant for gccgo/gollvm where some linkname 
references do not exist or work differently), so we can quickly edit the 
code and know where bugs are coming from. Many other packages that try to 
optimize json/cbor/msgpack encoding and decoding use some variation of the 
same ideas here.

I test the code  for the last 5 go releases on each github checkin, via 
travis CI. I also run with the standard compiler and gccgo locally before 
cutting a release. I have now added gollvm to my pre-release validation 
script, so it is validated before I cut the release. Caveat: I test with 
the installed versions of gccgo from ubuntu, and locally built gollvm. 
Since gollvm is not released yet, the version I test with may be old 
(building gollvm takes roughly 1 hour on my computer).

If you see any further issues, please file a bug and I will jump on 
it: https://github.com/ugorji/go/issues/new

Thanks.
On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:

> You haven't said whether you followed the "safe" instructions for 
> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, which 
> uses go:linkname.
>
> Package Documentation for github.com/ugorji/go/codec
> https://github.com/ugorji/go/blob/master/codec/README.md
>
> This package will carefully use 'package unsafe' for performance reasons 
> in specific places. You can build without unsafe use by passing the safe or 
> appengine tag i.e. 'go install -tags=codec.safe 
>
> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>
> go test -tags codec.safe -run Json
> go test -tags "alltests codec.safe" -run Suite
>
> Peter
>
> On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote:
>
>> >  go:linkname to refer directly to functions that are defined but not 
>> exported by the standard library. 
>> > This is not supported and is likely to break with any new release. It 
>> evidently breaks with GoLLVM.
>>
>> Thanks for your attention, but I tried to write a demo with go:linkname.
>> In fact, it works well with gollvm...So maybe it is not the exact cause 
>> for the problem
>>
>> Herei is my code:
>>
>> hello/hello.go
>> ```
>> package hello
>>
>> import (
>> "fmt"
>> _ "unsafe"
>> )
>> //go:linkname hellox hello.hellox
>> func hellox(x string) {
>> fmt.Println(x)
>> }
>> ```
>>
>> main.go
>> ```
>> package main
>>
>> import (
>> _ "mypackage/hello"
>> _ "unsafe"
>> )
>>
>> //go:linkname hel hello.hellox
>> func hel(x string)
>>
>> func main() {
>> hel("aaa")
>> //println("")
>> }
>> ```
>>
>> 在2021年3月29日星期一 UTC+8 上午12:37:49 写道：
>>
>>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote: 
>>> > 
>>> > For s2-geojson, I meet: 
>>> > 

[go-nuts] Re: political fundraising on golang.org!

[Ugorji Nwoke]

A simple message was posted: "Black Lives Matter. Support the Equal Justice 
Initiative".

I understand that this is controversial to say in some quarters. I am 
saddened that this is controversial to say here.

Rust developers succinctly captured why it is appropo to say in 
https://blog.rust-lang.org/2020/06/04/Rust-1.44.0.html, and I quote:
"The Rust Core Team believes that tech is and always will be political 
..."

If some renowned member of our go community were being publicly treated 
sub-par and golang.org posted a message of support for that member, no one 
would bat an eye. Let's have the same empathy here, and not split hairs on 
semantics. 

>From one proud black member of this community to the other members, we 
could really do without the public display of disapproval right now. It's a 
very sensitive time.

Thank you.


On Sunday, June 14, 2020 at 9:36:38 AM UTC-4, peterGo wrote:
>
> Recently, a political message with a fundraising link appeared as a banner 
> atop golang.org websites: https://golang.org/, https://pkg.go.dev/.
>
> content/static: add Black Lives Matter banner to top of site
> https://go-review.googlesource.com/c/website/+/237589
>
> 
> Black Lives Matter.
> https://support.eji.org/give/153413/#!/donation/checkout;
>target="_blank"
>rel="noopener">Support the Equal Justice Initiative.
> 
>
> How was this decision made?
>
> Go is a programming language. For political fundraising use personal 
> Twitter and Facebook accounts. 
>
> Peter
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0f206ac1-b34c-4726-a0ee-2db9afc79b02o%40googlegroups.com.

[go-nuts] Re: readonly []byte required

[Ugorji Nwoke]

This came up about 5 or so years ago.

Brad proposed it, it was deliberated for a while, and Russ articulated why 
it was no doable at the time. There may be a chance to resurrect it, but it 
will be good to see what was written before.

See

Prior discussion: 
https://groups.google.com/forum/?fromgroups=#!topic/golang-dev/Y7j4B2r_eDw
Proposal from Brad 
Fitzpatrick: 
https://docs.google.com/document/d/1UKu_do3FRvfeN5Bb1RxLohV-zBOJWTzX0E8ZU1bkqX0
Evaluation and response from 
Russ: 
https://docs.google.com/document/d/1-NzIYu0qnnsshMBpMPmuO21qd8unlimHgKjRD9qwp2A

Having said that, Russ put this on his list of items to think through for 
2017 (along with other items which are making it to Go 2 now). See 
https://research.swtch.com/go2017 . 

It may be worth it to see what his current thinking on that is.

+rsc @rsc 

On Thursday, July 4, 2019 at 10:27:58 PM UTC-4, Ally Dale wrote:
>
> []byte is mutable in Go, but string is readonly, so []byte(string) in Go 
> is actually allocate, and this operation is costly.
> Sometimes, we need a readonly []byte parameter but we have string only, 
> and we have no choice but allocate to make a new []byte variable 
> from string.
> Eg: net.Send([]byte("ping")) is costly, but net.Send(readonly 
> []byte("ping")) can be cheaply.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/72e8c24f-5cf6-4eaf-9b94-3afd066eb9b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Newbie question on slice bound checking

[Ugorji Nwoke]

Thanks much, to you and Jan Merci.

I needed it to validate whether the performance gap in my library was due 
to bounds checking or not. If it was, then I would try harder to use tips 
to reduce bounds checking. I wasn't planning on running it in production.

I ended up having to use -gcflags=all=-B (previously I was only using 
-gcflags=-B and I didn't see any improved performance for any packages).

On Sunday, June 16, 2019 at 9:34:17 AM UTC-4, Michael Jones wrote:
>
> It has been, yes, and unless it changed in the last few weeks, it still 
> is. 
>
> I use to to measure cost as an awareness metric, but seldom run that way. 
> Go is safe until you do that, then it becomes unsafe. Too risky for serious 
> use. 
>
> On Sun, Jun 16, 2019 at 4:53 AM Ugorji Nwoke  > wrote:
>
>> I know this is an old thread, but is -B still supported?
>>
>>
>> On Monday, August 27, 2012 at 2:09:19 AM UTC-4, minux wrote:
>>>
>>>
>>> On Monday, August 27, 2012, bluehorn wrote:
>>>>
>>>> Is "-gcflags -B" still supported in go1.0.2?  I suppose it was gone 
>>>> already.
>>>>
>>> It is still there. 
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/397c0824-b5b5-412d-974e-b53f56d29a82%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/397c0824-b5b5-412d-974e-b53f56d29a82%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
>
> *Michael T. jonesmichae...@gmail.com *
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/be4bb380-9e4d-45c4-a74e-7bce5b27f523%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Newbie question on slice bound checking

[Ugorji Nwoke]

I know this is an old thread, but is -B still supported?

On Monday, August 27, 2012 at 2:09:19 AM UTC-4, minux wrote:
>
>
> On Monday, August 27, 2012, bluehorn wrote:
>>
>> Is "-gcflags -B" still supported in go1.0.2?  I suppose it was gone 
>> already.
>>
> It is still there. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/397c0824-b5b5-412d-974e-b53f56d29a82%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is conversion between int and uint a no-op i.e. is it free

[Ugorji Nwoke]

Thanks Keith - this is what I was looking for.

On Saturday, November 24, 2018 at 6:49:33 PM UTC-5, Keith Randall wrote:
>
> int<->uint conversions should never generate any machine code. They are 
> free.
>
> On Saturday, November 24, 2018 at 10:55:50 AM UTC-8, Andy Balholm wrote:
>>
>> There is nothing in the language spec that guarantees anything about 
>> performance. But if logic tells you that it should be a no-op, and 
>> examination of the generated code shows you that it is a no-op in the cases 
>> you tested, you can safely assume that it is not going to be an issue for 
>> your program’s performance.
>>
>> Andy
>>
>> On Nov 24, 2018, at 8:45 AM, Ugorji Nwoke  wrote:
>>
>> Thanks so much Silviu. I love this tool - I had seen it before, but 
>> didn't realize it also supported go language. Thanks so much for bringing 
>> it up - it should help me do more investigation on my own faster.
>>
>> I used it to compare the asm output, and I got the same thing as when I 
>> did 
>> go build -gcflags "-S" num_conversion.go
>>
>> i.e. it leads me to conclude, as I suspected, that conversion from int to 
>> uint is free (no-op at runtime). 
>>
>> However, I get concerned that my proof may be insufficient, or there may 
>> be other reason why the asm looks same, and that is why I wanted a 
>> definitive answer from someone familiar with the internals.
>>
>>
>> On Saturday, November 24, 2018 at 11:28:43 AM UTC-5, Silviu Capota Mera 
>> wrote:
>>>
>>> A very nice tool from Matt Godbolt (and team of volunteers): 
>>> https://godbolt.org/z/4nt5cJ
>>>
>>> You can switch compiler version (e.g. Go 1.4, 1.7, 1.9, 1.11, tip, etc) 
>>> and/or gccgo, take a look at variations, etc
>>>
>>> On Saturday, 24 November 2018 11:07:51 UTC-5, Jan Mercl wrote:
>>>>
>>>> On Sat, Nov 24, 2018 at 4:31 PM Ugorji Nwoke  wrote:
>>>>
>>>> > Jan, you and I have the same understanding i.e. float <-> int is 
>>>> obviously non-free, but I can't think of why int <-> uint will not be 
>>>> free. 
>>>> However, I want someone with knowledge of the 
>>>>  > compiler/runtime/codegeneration/SSA internals that can give me a 
>>>> definitive answer. 
>>>>
>>>> Any correct compiler is an implementation of the language 
>>>> specification. From the language specification it follows that the 
>>>> compiler 
>>>> _may_ check that - for example - 42 != 314 or 278 == 278 while performing 
>>>> the 'uint' <-> 'int" conversion. It may also try to factor M4170639287. 
>>>> The 
>>>> question is why to do so when nothing of that is mandated by the language 
>>>> specification for a correct implementation?
>>>>
>>>> The next reasonable step is to assume Occam's razor is a thing.
>>>>
>>>> -- 
>>>>
>>>> -j
>>>>
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is conversion between int and uint a no-op i.e. is it free

[Ugorji Nwoke]

Thanks so much Silviu. I love this tool - I had seen it before, but didn't 
realize it also supported go language. Thanks so much for bringing it up - 
it should help me do more investigation on my own faster.

I used it to compare the asm output, and I got the same thing as when I did 
go build -gcflags "-S" num_conversion.go

i.e. it leads me to conclude, as I suspected, that conversion from int to 
uint is free (no-op at runtime). 

However, I get concerned that my proof may be insufficient, or there may be 
other reason why the asm looks same, and that is why I wanted a definitive 
answer from someone familiar with the internals.


On Saturday, November 24, 2018 at 11:28:43 AM UTC-5, Silviu Capota Mera 
wrote:
>
> A very nice tool from Matt Godbolt (and team of volunteers): 
> https://godbolt.org/z/4nt5cJ
>
> You can switch compiler version (e.g. Go 1.4, 1.7, 1.9, 1.11, tip, etc) 
> and/or gccgo, take a look at variations, etc
>
> On Saturday, 24 November 2018 11:07:51 UTC-5, Jan Mercl wrote:
>>
>> On Sat, Nov 24, 2018 at 4:31 PM Ugorji Nwoke  wrote:
>>
>> > Jan, you and I have the same understanding i.e. float <-> int is 
>> obviously non-free, but I can't think of why int <-> uint will not be free. 
>> However, I want someone with knowledge of the 
>>  > compiler/runtime/codegeneration/SSA internals that can give me a 
>> definitive answer. 
>>
>> Any correct compiler is an implementation of the language specification. 
>> From the language specification it follows that the compiler _may_ check 
>> that - for example - 42 != 314 or 278 == 278 while performing the 'uint' 
>> <-> 'int" conversion. It may also try to factor M4170639287. The question 
>> is why to do so when nothing of that is mandated by the language 
>> specification for a correct implementation?
>>
>> The next reasonable step is to assume Occam's razor is a thing.
>>
>> -- 
>>
>> -j
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is conversion between int and uint a no-op i.e. is it free

[Ugorji Nwoke]

Jan, you and I have the same understanding i.e. float <-> int is obviously 
non-free, but I can't think of why int <-> uint will not be free. However, 
I want someone with knowledge of the compiler/runtime/codegeneration/SSA 
internals that can give me a definitive answer. 

On Saturday, November 24, 2018 at 10:25:58 AM UTC-5, Jan Mercl wrote:
>
> On Sat, Nov 24, 2018 at 4:02 PM Ugorji Nwoke  > wrote:
>
> > I understand the rules from the context of the language and what the 
> compiler will accept - conversion MUST be explicit. I am asking if there is 
> any runtime cost to the conversion between int and uint,
> > given that they "should" have the same representation, or if it is free. 
> I can see how there is a cost for float64 <-> float32, or for float64 <-> 
> int, but is there a cost for the specific case of int <-> uint?
>
> 'floatxx' <-> 'floatyy' <-> 'intzz' is obviously non free.
>
> But what possible run-time cost in conversion between 'uint' and 'int' - 
> considering it's always valid and the bit representations have the same 
> shape, size, organization and modulo the sign bit also interpretation - can 
> you think of?
>
>
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is conversion between int and uint a no-op i.e. is it free

[Ugorji Nwoke]

Thanks Jan. 

But my question was different. 

I understand the rules from the context of the language and what the 
compiler will accept - conversion MUST be explicit. I am asking if there is 
any runtime cost to the conversion between int and uint, given that they 
"should" have the same representation, or if it is free. I can see how 
there is a cost for float64 <-> float32, or for float64 <-> int, but is 
there a cost for the specific case of *int <-> uint*?

i.e. if I do a lot of calls like:

var cursor uint = 4
var b []byte
if curson < uint(len(b)) { ... }

Is there a runtime cost to that uint(len(b)), or no?

On Saturday, November 24, 2018 at 9:57:06 AM UTC-5, Jan Mercl wrote:
>
> On Sat, Nov 24, 2018 at 3:43 PM Ugorji Nwoke  > wrote:
>
> Here is the authoritative <https://golang.org/ref/spec#Conversions> 
> answer:
>
> """"
> A non-constant value x can be converted to type T in any of these cases:
> - x is assignable <https://golang.org/ref/spec#Assignability> to 
> T.
> - ignoring struct tags (see below), x's type and T have identical 
> <https://golang.org/ref/spec#Type_identity> underlying types 
> <https://golang.org/ref/spec#Types>.
> - ignoring struct tags (see below), x's type and T are pointer 
> types that are not defined types 
> <https://golang.org/ref/spec#Type_definitions>, and their pointer base 
> types have identical underlying types.
> - x's type and T are both integer or floating point types.
> - x's type and T are both complex types.
> - x is an integer or a slice of bytes or runes and T is a string 
> type.
> - x is a string and T is a slice of bytes or runes.
> """"
>
> From the above it follows that your case of converting between 'uint' and 
> 'int' values at runtime is always valid due to: "x's type and T are both 
> integer or floating point types."
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] are interface calls to empty methods optimized away - so there is no function call overhead beyond the dereference/jump?

[Ugorji Nwoke]

Yes. Fundamentally, I'm asking if step #2 in your calling sequence can be 
optimized. 

1. load a function pointer from the itab 
2. call the function pointer. *However, if the function pointer to is 
empty, just do a NOP here and skip the function-call-dance. This should be 
possible as the i-tab is created dynamically on first reference, which 
means that it can know if the function is empty and put NOP indicator in 
the i-tab instead of the function pointer. *That's the optimization I'm 
asking about. 

On Friday, December 1, 2017 at 2:22:24 PM UTC-5, Ian Lance Taylor wrote:
>
> On Fri, Dec 1, 2017 at 10:42 AM, Ugorji Nwoke <ugo...@gmail.com 
> > wrote: 
> > 
> > This is not so much about de-virtualization. It's about not making a 
> call to 
> > an empty function i.e. I expect that interface calls come down to 
> > 1. dereference the pointer to the i-tab, 
> > 2. jump to the appropriate function in the i-tab, 
> > 3. call that function. 
>
> That does not seem to me to be an accurate description of the current 
> implementation.  Calling a method of an interface is 
> 1. load a function pointer from the itab 
> 2. call the function pointer 
>
>
> > If that function is a no-op, is the runtime smart enough to skip that 
> > function call completely (e.g. replacing the entry in the i-tab with a 
> NOP)? 
>
> The itab does not contain code, it just holds function pointers. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] are interface calls to empty methods optimized away - so there is no function call overhead beyond the dereference/jump?

[Ugorji Nwoke]

I did, but couldn't infer my answer out of the assembly. I see the
following below, but it's not clear if the call is made, or if the linker
does some optimization after the compiler step. See below:

"".T1.Do STEXT nosplit size=1 args=0x8 locals=0x0
0x 0 (check-interface-noop-call.go:15) TEXT "".T1.Do(SB), NOSPLIT,
$0-8
0x 0 (check-interface-noop-call.go:15) FUNCDATA $0,
gclocals·2a5305abe05176240e61b8620e19a815(SB)
0x 0 (check-interface-noop-call.go:15) FUNCDATA $1,
gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
0x 0 (check-interface-noop-call.go:15) RET
0x c3   .
"".main STEXT size=84 args=0x0 locals=0x18
0x 0 (check-interface-noop-call.go:19) TEXT "".main(SB), $24-0
0x 0 (check-interface-noop-call.go:19) MOVQ (TLS), CX
0x0009 9 (check-interface-noop-call.go:19) CMPQ SP, 16(CX)
0x000d 00013 (check-interface-noop-call.go:19) JLS 77
0x000f 00015 (check-interface-noop-call.go:19) SUBQ $24, SP
0x0013 00019 (check-interface-noop-call.go:19) MOVQ BP, 16(SP)
0x0018 00024 (check-interface-noop-call.go:19) LEAQ 16(SP), BP
0x001d 00029 (check-interface-noop-call.go:19) FUNCDATA $0,
gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
0x001d 00029 (check-interface-noop-call.go:19) FUNCDATA $1,
gclocals·33cdeebe80329f1fdbee7f5874cb(SB)
0x001d 00029 (check-interface-noop-call.go:22) LEAQ go.itab."".T1,"".I(SB),
AX
0x0024 00036 (check-interface-noop-call.go:22) TESTB AL, (AX)
0x0026 00038 (check-interface-noop-call.go:22) MOVQ go.itab."".T1,"".I+24(SB),
AX
0x002d 00045 (check-interface-noop-call.go:22) MOVQ $1, 8(SP)
0x0036 00054 (check-interface-noop-call.go:22) LEAQ runtime.zerobase(SB), CX
0x003d 00061 (check-interface-noop-call.go:22) MOVQ CX, (SP)
0x0041 00065 (check-interface-noop-call.go:22) PCDATA $0, $0
0x0041 00065 (check-interface-noop-call.go:22) CALL AX
0x0043 00067 (check-interface-noop-call.go:26) MOVQ 16(SP), BP
0x0048 00072 (check-interface-noop-call.go:26) ADDQ $24, SP
0x004c 00076 (check-interface-noop-call.go:26) RET
0x004d 00077 (check-interface-noop-call.go:26) NOP
0x004d 00077 (check-interface-noop-call.go:19) PCDATA $0, $-1
0x004d 00077 (check-interface-noop-call.go:19) CALL
runtime.morestack_noctxt(SB)
0x0052 00082 (check-interface-noop-call.go:19) JMP 0


On Fri, Dec 1, 2017 at 1:59 PM, Michael Banzon <mich...@banzon.dk> wrote:

> I don’t have specific knowledge about how this works - but wouldn’t it be
> fairly easy to test by having the compiler emit the assembly code?
>
> --
> Michael Banzon
> https://michaelbanzon.com/
>
>
>
>
> Den 1. dec. 2017 kl. 19.42 skrev Ugorji Nwoke <ugo...@gmail.com>:
>
> Thanks Ian.
>
> This is not so much about de-virtualization. It's about not making a call
> to an empty function i.e. I expect that interface calls come down to
> 1. dereference the pointer to the i-tab,
> 2. jump to the appropriate function in the i-tab,
> 3. call that function.
>
> If that function is a no-op, is the runtime smart enough to skip that
> function call completely (e.g. replacing the entry in the i-tab with a
> NOP)? That way, making an interface call to an empty function becomes just
> a dereference+jump, and *not* dereference+jump+(unnecessary)func-call ?
>
> Currently, in my code, within a tight loop, i have a lot of:
> for {
> if v.HasSeparators() { v.ReadMapElem() }
> ...
> if v.HasSeparators() { v.ReadMapEnd() }
> ...
> }
>
> If the func call is elided, then I will just call the empty function all
> the time and remove the if/conditional checks. The code will become:
> for {
> v.ReadMapElem()
> ...
> v.ReadMapEnd()
> ...
> }
>
>
> On Friday, December 1, 2017 at 10:18:38 AM UTC-5, Ian Lance Taylor wrote:
>>
>> On Fri, Dec 1, 2017 at 2:40 AM, Ugorji Nwoke <ugo...@gmail.com> wrote:
>> >
>> > I know that a no-op function call is optimized away, as it is inlined
>> to
>> > nothing.
>> >
>> > However, what about a no-op interface call?
>> >
>> > See sample code:
>> >
>> > type I interface { Do(int) }
>> > type T1 struct{}
>> > func (_ T1) Do(i int) {}
>> > func main() {
>> > var v I
>> > v = T1{}
>> > v.Do(1)
>> > }
>> >
>> > Is it safe to assume the following that calling T1.Do(...) via an
>> interface
>> > costs a dereference + jump (but NO function call overhead)?
>> >
>> > I currently have code where I do a lot of conditional checks to
>> determine
>> > whether to make that interface call or not.
>> > However, if I know that the no function call overhead is done when the
>> > dynami

Re: [go-nuts] are interface calls to empty methods optimized away - so there is no function call overhead beyond the dereference/jump?

[Ugorji Nwoke]

Thanks Ian.

This is not so much about de-virtualization. It's about not making a call 
to an empty function i.e. I expect that interface calls come down to 
1. dereference the pointer to the i-tab, 
2. jump to the appropriate function in the i-tab, 
3. call that function. 

If that function is a no-op, is the runtime smart enough to skip that 
function call completely (e.g. replacing the entry in the i-tab with a 
NOP)? That way, making an interface call to an empty function becomes just 
a dereference+jump, and *not* dereference+jump+(unnecessary)func-call ? 

Currently, in my code, within a tight loop, i have a lot of:
for {
if v.HasSeparators() { v.ReadMapElem() }
...
if v.HasSeparators() { v.ReadMapEnd() }
... 
}

If the func call is elided, then I will just call the empty function all 
the time and remove the if/conditional checks. The code will become:
for {
v.ReadMapElem()
...
v.ReadMapEnd() 
...
}


On Friday, December 1, 2017 at 10:18:38 AM UTC-5, Ian Lance Taylor wrote:
>
> On Fri, Dec 1, 2017 at 2:40 AM, Ugorji Nwoke <ugo...@gmail.com 
> > wrote: 
> > 
> > I know that a no-op function call is optimized away, as it is inlined to 
> > nothing. 
> > 
> > However, what about a no-op interface call? 
> > 
> > See sample code: 
> > 
> > type I interface { Do(int) } 
> > type T1 struct{} 
> > func (_ T1) Do(i int) {} 
> > func main() { 
> > var v I 
> > v = T1{} 
> > v.Do(1) 
> > } 
> > 
> > Is it safe to assume the following that calling T1.Do(...) via an 
> interface 
> > costs a dereference + jump (but NO function call overhead)? 
> > 
> > I currently have code where I do a lot of conditional checks to 
> determine 
> > whether to make that interface call or not. 
> > However, if I know that the no function call overhead is done when the 
> > dynamic function is a no-op, then I will just call the interface 
> > method all the time, and not try to be smart within code. 
> > 
> > I tried looking at "go tool compile -S main.go" output, but I still see 
> a 
> > CALL in there, but don't know whether the linker/compiler/something 
> > optimizes this out. 
>
> In general, no.  The compiler doesn't currently try to figure out the 
> dynamic type of an interface (an optimization known as 
> devirtualization).  An interface method can only be inlined when the 
> dynamic type is known. 
>
> The cost of calling an empty method is quite low.  It's not zero, but 
> I am skeptical that the cost of the call is more than the cost of the 
> conditional checks you mention. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] are interface calls to empty methods optimized away - so there is no function call overhead beyond the dereference/jump?

[Ugorji Nwoke]

I know that a no-op function call is optimized away, as it is inlined to 
nothing.

However, what about a no-op interface call?

See sample code:

type I interface { Do(int) }
type T1 struct{}
func (_ T1) Do(i int) {}
func main() {
var v I
v = T1{}
v.Do(1)
}

Is it safe to assume the following that calling T1.Do(...) via an interface 
costs a dereference + jump (but NO function call overhead)?

I currently have code where I do a lot of conditional checks to determine 
whether to make that interface call or not. 
However, if I know that the no function call overhead is done when the 
dynamic function is a no-op, then I will just call the interface
method all the time, and not try to be smart within code.

I tried looking at "go tool compile -S main.go" output, but I still see a 
CALL in there, but don't know whether the linker/compiler/something
optimizes this out.

Please explain.

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Running multiple iterations of "go test" in a single process, to allow for cumulative coverage results

[Ugorji Nwoke]

never mind - I figured it out. 

On Tuesday, September 12, 2017 at 6:08:27 PM UTC-4, Ugorji Nwoke wrote:
>
> I currently use command line parameters to test that various options work 
> fine.
>
> For example, I define some test flags like -tv -ta -ti ... that set 
> different options on my test, and then run my tests.
>
> The execution looks like this:
>
> go test -tv
> go test -ti
> go test -ta 
> ...
>
> Unfortunately, I can't generate accurate coverage results with this.
>
> I would like a way to call all the different permutations in a single "go 
> test " call. 
>
> Any ideas on how to do this?
>
> Thanks.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Running multiple iterations of "go test" in a single process, to allow for cumulative coverage results

[Ugorji Nwoke]

I currently use command line parameters to test that various options work 
fine.

For example, I define some test flags like -tv -ta -ti ... that set 
different options on my test, and then run my tests.

The execution looks like this:

go test -tv
go test -ti
go test -ta 
...

Unfortunately, I can't generate accurate coverage results with this.

I would like a way to call all the different permutations in a single "go 
test " call. 

Any ideas on how to do this?

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: marshall/unmarshall of large json files

[Ugorji Nwoke]

golang's encoding/json package likes to ensure that the 
1. you read 12MB into memory
2. encoding/json will first scan the full text to ensure it is a 
well-formed json (before populating the structure)
3. At a later point, GC will reclaim that memory
4. This means that if each GET request is doing this 12MB decode, you will 
start getting memory pressure.

I think this is a design problem more so. You can help with #1 and #3 and 
#4 above: do not read up the 12MB into memory, but instead use a 
json.NewDecoder(os.OpenFile(...)).Decode(...). However, to tackle #2, you 
may need to examine a different decoder. Try package: github.com/ugorji/go 
( https://godoc.org/github.com/ugorji/go/codec ).

On Monday, November 7, 2016 at 5:49:45 PM UTC-5, Gokul Muthuswamy wrote:
>
> I have created a mock with a test file the code attached 
> jsoncodegolang.txt, attached is also a test file to simulate our data 
> entries. In the real world scenario its around 12MB pull on every GET 
> operation.
>
>
>
> On Mon, Nov 7, 2016 at 1:02 PM, Ugorji Nwoke <ugo...@gmail.com 
> > wrote:
>
>> Show some code, and we may be able to advise you better.
>>
>>
>> On Monday, November 7, 2016 at 1:26:49 PM UTC-5, gmuth...@gmail.com 
>> wrote:
>>>
>>> We are using Go for one of our projects and our API's use JSON as the 
>>> payload formatter.  We have a large json file (12MB) that needs to be 
>>> unmarshalled. We just used the standard json enconde/decode and it bloats 
>>> up the memory. I ran the pprof an realized it was due to 
>>> reflect_unsafe.NewArray that needs a lot of allocation.  Is this the 
>>> standard way to implement API's with large json files in Go or should I not 
>>> use encode/decode for this purpose?
>>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: marshall/unmarshall of large json files

[Ugorji Nwoke]

Show some code, and we may be able to advise you better.


On Monday, November 7, 2016 at 1:26:49 PM UTC-5, gmuth...@gmail.com wrote:
>
> We are using Go for one of our projects and our API's use JSON as the 
> payload formatter.  We have a large json file (12MB) that needs to be 
> unmarshalled. We just used the standard json enconde/decode and it bloats 
> up the memory. I ran the pprof an realized it was due to 
> reflect_unsafe.NewArray that needs a lot of allocation.  Is this the 
> standard way to implement API's with large json files in Go or should I not 
> use encode/decode for this purpose?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Search for Another JSON Package

[Ugorji Nwoke]

https://godoc.org/github.com/ugorji/go/codec
https://github.com/ugorji/go
package name: github.com/ugorji/go/codec

On Wednesday, August 24, 2016 at 5:30:10 AM UTC-4, dc0d wrote:
>
> Is there a JSON package that have these characteristics?
>
>
>- can marshal numeric integer values to strings (like 
>using `json:",string"` tag)
>- can unmarshal  numeric integer values from *either* string or the 
>number; based on the type of the field (some field like `MessageID int64`)
>
> Why? Because IEEE 754 is really annoying and JSON standard treats all 
> numeric values as `float64`. So when a big integer number (a big `int64` 
> for example) gets serialized - which does perfectly - it can not get 
> deserialized.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.