Than, Appreciate you saying that. And you're very welcome.
On Tue, Mar 30, 2021 at 8:29 PM Than McIntosh <th...@google.com> wrote: > Thank you @Ugorji Nwoke <ugo...@google.com> , I appreciate your going > the extra mile to include gollvm in your testing. > > Cheers, Than > > > On Tue, Mar 30, 2021 at 11:17 AM Ugorji Nwoke <ugo...@gmail.com> wrote: > >> Ugorji here - author of the github.com/ugorji/go/codec >> <https://pkg.go.dev/github.com/ugorji/go/codec> package. >> >> The notes below are for folks that are interested in why we use unsafe, >> and how we mitigate concerns around it. >> >> As Peter mentioned above, you can pass the build tag "codec.safe" to >> bypass using unsafe code where we try to reach deeper into the runtime >> implementation to optimize. >> >> I have been supporting gccgo for a while now, but never tested with >> gollvm, as it has not been released at this time. I built a gollvm version >> yesterday (from source) and tested it, and made some changes (see here >> <https://github.com/ugorji/go/commit/1768cf4d4babceec77d8ab0da478c58c9cb9aa99> >> and here >> <https://github.com/ugorji/go/commit/12f7e67b2326c6cc05bd3fea410164a18108d6a3>) >> so that gollvm will work in the default (high-performance) mode. I plan to >> cut a new release v1.2.5 sometime this week with those changes. >> >> For those who care about why we support unsafe and dig into the runtime >> internals, please read below. >> >> To illustrate the benefit, look at unsafe vs codec.safe benchmark results >> below: >> ``` >> ---- tags: "" (default high-performance mode using unsafe carefully) ---- >> Benchmark__Json_______Encode-8 6921 152808 ns/op 24 B/op >> 1 allocs/op >> Benchmark__Json_______Encode-8 5622 197863 ns/op 10048 B/op >> 381 allocs/op >> >> ---- tags: "codec.safe" ---- >> Benchmark__Json_______Decode-8 2587 415595 ns/op 71878 B/op >> 592 allocs/op >> Benchmark__Json_______Decode-8 2167 478122 ns/op 96812 B/op >> 1456 allocs/op >> >> ---- tags: x generated ---- >> Benchmark__Json_______Encode-8 8694 120519 ns/op 1528 B/op >> 6 allocs/op >> Benchmark__Json_______Decode-8 2960 349320 ns/op 70469 B/op >> 589 allocs/op >> ``` >> >> This benchmarks show that using unsafe carefully can cut down allocations >> dramatically. Encoding allocation goes from 381 to 1, while decoding goes >> from 1456 to 592. Those numbers using unsafe rival the allocation numbers >> that we get using code-generation (as seen above), and the run time starts >> to trend within 25% of the code-generation numbers, and 25% better than the >> non-unsafe run time. >> >> We limited the surface area that is exposed to unsafe (basically 1 file, >> helper_unsafe.go, with some variant for gccgo/gollvm where some linkname >> references do not exist or work differently), so we can quickly edit the >> code and know where bugs are coming from. Many other packages that try to >> optimize json/cbor/msgpack encoding and decoding use some variation of the >> same ideas here. >> >> I test the code for the last 5 go releases on each github checkin, via >> travis CI. I also run with the standard compiler and gccgo locally before >> cutting a release. I have now added gollvm to my pre-release validation >> script, so it is validated before I cut the release. Caveat: I test with >> the installed versions of gccgo from ubuntu, and locally built gollvm. >> Since gollvm is not released yet, the version I test with may be old >> (building gollvm takes roughly 1 hour on my computer). >> >> If you see any further issues, please file a bug and I will jump on it: >> https://github.com/ugorji/go/issues/new >> >> Thanks. >> On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote: >> >>> You haven't said whether you followed the "safe" instructions for >>> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, >>> which uses go:linkname. >>> >>> Package Documentation for github.com/ugorji/go/codec >>> https://github.com/ugorji/go/blob/master/codec/README.md >>> >>> This package will carefully use 'package unsafe' for performance reasons >>> in specific places. You can build without unsafe use by passing the safe or >>> appengine tag i.e. 'go install -tags=codec.safe >>> >>> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g. >>> >>> go test -tags codec.safe -run Json >>> go test -tags "alltests codec.safe" -run Suite >>> >>> Peter >>> >>> On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote: >>> >>>> > go:linkname to refer directly to functions that are defined but not >>>> exported by the standard library. >>>> > This is not supported and is likely to break with any new release. It >>>> evidently breaks with GoLLVM. >>>> >>>> Thanks for your attention, but I tried to write a demo with go:linkname. >>>> In fact, it works well with gollvm...So maybe it is not the exact cause >>>> for the problem >>>> >>>> Herei is my code: >>>> >>>> hello/hello.go >>>> ``` >>>> package hello >>>> >>>> import ( >>>> "fmt" >>>> _ "unsafe" >>>> ) >>>> //go:linkname hellox hello.hellox >>>> func hellox(x string) { >>>> fmt.Println(x) >>>> } >>>> ``` >>>> >>>> main.go >>>> ``` >>>> package main >>>> >>>> import ( >>>> _ "mypackage/hello" >>>> _ "unsafe" >>>> ) >>>> >>>> //go:linkname hel hello.hellox >>>> func hel(x string) >>>> >>>> func main() { >>>> hel("aaa") >>>> //println("aaaa") >>>> } >>>> ``` >>>> >>>> 在2021年3月29日星期一 UTC+8 上午12:37:49<Ian Lance Taylor> 写道： >>>> >>>>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙 <f011...@gmail.com> wrote: >>>>> > >>>>> > For s2-geojson, I meet: >>>>> > ``` >>>>> > # github.com/pantrif/s2-geojson/cmd/s2-geojson >>>>> > >>>>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function >>>>> github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined >>>>> reference to 'reflect.unsafe_New' >>>>> > >>>>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function >>>>> github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: >>>>> error: >>>>> undefined reference to 'reflect.unsafe_New' >>>>> > decode.go:509: error: undefined reference to 'reflect.unsafe_New' >>>>> > decode.go:14935: error: undefined reference to 'reflect.unsafe_New' >>>>> >>>>> This is a problem with github.com/ugorji/go. The file >>>>> code/helper_unsafe.go uses go:linkname to refer directly to functions >>>>> that are defined but not exported by the standard library. This is >>>>> not supported and is likely to break with any new release. It >>>>> evidently breaks with GoLLVM. >>>>> >>>>> Ian >>>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/584aa92a-8528-45a7-870c-9153298e95adn%40googlegroups.com >> <https://groups.google.com/d/msgid/golang-nuts/584aa92a-8528-45a7-870c-9153298e95adn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- Ugorji Nwoke ugo...@google.com Tel: +1-202-750-4567 Technology Account Executive, Retail & Hospitality -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAKBA3t%2Bxk%2BbEDAUtC81x%2BrF2fFVcCvEVfNcVfE-smYV_tEg7oA%40mail.gmail.com.
