On Tuesday, September 24, 2019 at 2:44:46 AM UTC-7, fra...@adeven.com wrote: > > Robert Engels, I am not familiar with the two libraries you named. But > from your description I think (I'm not sure) that we have different uses in > mind. > > The escape analysis that would be required for us to avoid using unsafe is > _possible_, but does not yet exist in the Go compiler. The compiler > facilities needed to negate our need for unsafe are best described in this > issue > > https://github.com/golang/go/issues/2205 > > So the pattern that exists in many Go programs is that you have a variable > of type string([]byte) and you have a function which takes type > []byte(string) and the function only reads from []byte(string) argument. > > Often this comes in the form of 'I have a string, and I want to write its > contents via some interface which takes []byte. Most often it is about > code-deduplication, we want to read from some kind of sequence of bytes and > []byte and string would both do fine. > > While this does open us up to a class of bugs, which are both dangerous > and potentially hard to diagnose the places I see it used are usually very > self contained and the benefits, if that read string/[]byte lies on a hot > path are potentially significant. > > Although I don't know what UnsafeString/SafeString are used for I > _suspect_ they are for high-performance unsafe manipulation of strings. I > have never seen anyone try to use unsafe to do this in Go. Someone probably > does, but the overwhelmingly most common use case that I see is 'turn this > string into a []byte and use this function to read from it' going the other > way is less common. > > Personally I would _love_ to see the read-only bytes escape analysis built > into the compiler so we can throw away all of this unsafe code. > > I would really love that also. I think we could do it for some cases, by propagating information about writes to []byte args up the call tree. Unfortunately, probably the most common case would be passing through an interface (io.Writer), which would defeat that analysis.
> On Monday, September 23, 2019 at 6:38:27 PM UTC+2, Robert Engels wrote: >> >> As someone that has worked with a lot of similar libraries in the HFT >> space - things like UnsafeString or FastString in Java I would caution >> against doing this in Go - especially as proposed here. Taking an immutable >> object like string and making it mutable by accident is a recipe for >> disaster. You are almost always better mapping a struct with accessors and >> letting Go escape analysis perform the work on the stack and keep the >> safety. >> >> >> >> On Sep 23, 2019, at 10:09 AM, Francis <francis...@gmail.com> wrote: >> >> So I think the current state of unsafe conversions of string <-> []byte >> is roughly >> >> 1. Use the reflect Slice/StringHeader struct. These structs give you >> clear fields to set and read from. If the runtime representation of a >> string or []byte ever changes then these structs should change to reflect >> this (they have a non-backwards compatibility carve out in the comments). >> But this also means that you run into all these exotic problems because >> these two structs have a `uintpr` an `unsafe.Pointer` so for a short time >> the GC won't realise you are reading/writing a pointer. This makes correct >> use of these structs very difficult. >> 2. You can just cast between these two types going through >> `unsafe.Pointer` on the way. This works, because these two types have >> almost identical layouts. We don't use any uintptr at all and so the GC >> probably won't get confused. But, if the representations of string or >> []byte ever change then you code breaks silently, and could have very >> weird/hard to track down problems. >> >> So I don't think `neither is safer than the other` is quite the right >> description in this context. They both have problems, so they are both >> not-perfect. But their problems are quite distinct. At the least if we >> choose one over the other we can describe clearly which set of problems we >> want to have. >> >> My hope was that someone had thought through these problems and could >> indicate the right way to do it. >> >> On a related note. I was trying to track down where the >> Slice/StringHeader was first introduced. It was a long time ago >> >> <Rob Pike> (10 years ago) 29e6eb21ec (HEAD) >> >> make a description of the slice header public >> >> R=rsc >> DELTA=18 (3 added, 0 deleted, 15 changed) >> OCL=31086 >> CL=31094 >> >> Although I couldn't open that CL in gerrit (I assume user-error). From >> reading the commit I think the intention was for these header structs to be >> used for this or similar things. But the data was represented as a uintptr >> and a comment explicitly states that these structs are of no use without >> `unsafe.Pointer`. I have seen roughly three other CL which try to change >> the data field to `unsafe.Pointer` but are rejected because they change the >> reflect packages API. >> >> There is also this issue >> >> https://github.com/golang/go/issues/19367 >> >> Which proposes that Slice/StringHeader be moved/duplicated in unsafe and >> use `unsafe.Pointer`. As far as I can tell once we have this then all the >> subtle problems disappear and lovingly crafted examples like >> >> https://github.com/m3db/m3x/blob/master/unsafe/string.go#L62 >> >> just become the right way to do it. >> >> Until then maybe we should just rely on the structural similarities >> between the two types and cast between them. This seems especially >> appealing as Jan pointed out above that at least one of the hypothetical >> problems isn't hypothetical at all. >> >> >> On Monday, 23 September 2019 12:43:34 UTC+2, kortschak wrote: >>> >>> Any particular reason for that? Neither is safer than the other and >>> it's not clear to me that you can actually achieve the goal of having a >>> compile-time check for the correctness of this type of conversion. >>> >>> On Mon, 2019-09-23 at 02:36 -0700, fra...@adeven.com wrote: >>> > But this relies on a string's representation being the same as, but a >>> > bit smaller thabn, a []byte. I would prefer to use >>> > the Slice/StringHeader. >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golan...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c9d7f8a4-a65b-4dbb-995b-4274631387a1%40googlegroups.com.
