Re: [go-nuts] Understanding how []byte("astring") works

[Shulhan]

On Mon, 14 Jun 2021 10:24:00 +1000
Amit Saha  wrote:

> Hi - My main motivation to understand this is i always had to google
> this - how to convert a string to a byte slice.
> 
> Is []byte a type that has been defined in the compiler?
> 
> Or, is that an internal level detail that an earlier stage (parsing)
> takes care of when the compiler sees that statement?
> 
> Thanks,
> Amit
> 

Maybe this blogs can help: https://blog.golang.org/strings

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20210614194026.329ce8b9%40inspiro.localdomain.


pgpfSG6EONjHk.pgp
Description: OpenPGP digital signature

Re: [go-nuts] Understanding how []byte("astring") works

[Amit Saha]

On Mon, Jun 14, 2021 at 3:23 PM Roland Müller  wrote:
>
> Hello,
>
> Am Mo., 14. Juni 2021 um 03:24 Uhr schrieb Amit Saha :
>>
>> Hi - My main motivation to understand this is i always had to google
>> this - how to convert a string to a byte slice.
>>
>> Is []byte a type that has been defined in the compiler?
>>
>> Or, is that an internal level detail that an earlier stage (parsing)
>> takes care of when the compiler sees that statement?
>>
>> Thanks,
>> Amit
>>
>
> a []byte is a sequence of octets and strings in Go consistent of a bytes. 
> These bytes represent an sequence of unicode characters according to UTF-8. 
> One such character consists of either a single or two bytes. ASCII -only 
> strings than have as many bytes as UTF characters.
>
> In the example I made two loop functions loopStringByBytes(s) and 
> loopStringByChars(s) and checked them against a ASCII string and a cyrillic 
> string. You can see that for second string every character occupies 2 bytes.
>
> https://play.golang.org/p/DDSpiFuR8Lp

Thank you.

>
> BR,
> Roland
>
>
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CANODV3nBYmshDLFwUUdnnyVuvpjhWnwBJOb%3DwrZKEHXmtBgbSg%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANODV3k6XpFuP943bC7CCT9nnnFz5J_-trL%3D7rAgowsgWZj%3Drg%40mail.gmail.com.

Re: [go-nuts] Understanding how []byte("astring") works

[Amit Saha]

On Mon, Jun 14, 2021 at 4:44 PM Axel Wagner
 wrote:
>
> Hi,
>
> On Mon, Jun 14, 2021 at 2:24 AM Amit Saha  wrote:
>>
>> Hi - My main motivation to understand this is i always had to google
>> this - how to convert a string to a byte slice.
>>
>> Is []byte a type that has been defined in the compiler?
>
>
> No, but `byte` is. It is a predeclared identifier, referring to a builtin 
> integer type (it's an alias for `uint8`).
> So the compiler knows what a `byte` is and what a slice is, so it knows what 
> a slice of byte is.
>
> The conversion between a slice of `byte`s and a `string` is then defined in 
> the spec:
>
>> A non-constant value x can be converted to type T in any of these cases:
>>
>> […]
>>
>> x is an integer or a slice of bytes or runes and T is a string type.
>>
>> x is a string and T is a slice of bytes or runes.
>
> This means the compiler, when seeing a conversion, explicitly tests for these 
> two cases.
>
> How the conversion then actually works, you can see in the compiler explorer. 
> The compiler emits a call into a function called `runtime.stringtoslicebyte`.
> You can actually find that function in the source code of the runtime then 
> (the same function also contains other functions implementing similar 
> conversions).
> Really, it just allocates a new `[]byte` of the appropriate size and then 
> copies over the bytes from the string.
>
>> Or, is that an internal level detail that an earlier stage (parsing)
>
>
> You can actually mess with this a little bit to show that it's not done in 
> the parsing stage, but that the compiler actually knows if a type is a slice 
> of bytes or not.
> Because `byte` is a predeclared identifier, it can be shadowed, like every 
> other identifier. That is, you can declare your *own* type called `byte`:
> https://play.golang.org/p/6vDjw9eWX9s
> You can also define your own alias for `uint8` and then convert that:
> https://play.golang.org/p/nvseU7ofRru
> Or you can do both - first shadown the builtin `byte` alias and *then* create 
> your own:
> https://play.golang.org/p/1Y80stIxa5v
> You can also define your own type called `uint8` and then define `byte` as an 
> alias to that and see that you can no longer convert:
> https://play.golang.org/p/xUQrAmMm5Km
>
> All of this shows that the compiler really can't just rely on parsing and the 
> name. It really needs to have a notion of whether something is a slice of a 
> specific pre-declared type `uint8`, no matter what it is called in the source 
> code.
>
> It does that by creating a "virtual" package builtin inside the compiler and 
> then synthesizing type-definitions in that package. The code for that is here.
> But it should be noted that this package doesn't really exist and behaves 
> significantly different from "normal" packages - not just because it is 
> implemented entirely in the compiler/runtime, but also because it's 
> identifier are defined "outside" any package, in the universe block.
>
> I assume this covers all questions :) Let us know if you have follow-ups :)

Thank you! That's enough for me to start digging in for a bit.
>
> Axel
>
>> takes care of when the compiler sees that statement?
>>
>> Thanks,
>> Amit
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CANODV3nBYmshDLFwUUdnnyVuvpjhWnwBJOb%3DwrZKEHXmtBgbSg%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANODV3nw-RH1wdj4Zx2%2BY1UMojZSjFrXFBy_MA3rvmC-vW9bTQ%40mail.gmail.com.

Re: [go-nuts] Understanding how []byte("astring") works

['Axel Wagner' via golang-nuts]

Hi,

On Mon, Jun 14, 2021 at 2:24 AM Amit Saha  wrote:

> Hi - My main motivation to understand this is i always had to google
> this - how to convert a string to a byte slice.
>
> Is []byte a type that has been defined in the compiler?
>

No, but `byte` is. It is a predeclared identifier
, referring to a
builtin integer type  (it's an
alias for `uint8`).
So the compiler knows what a `byte` is and what a slice is, so it knows
what a slice of byte is.

The conversion between a slice of `byte`s and a `string` is then defined in
the spec :

A non-constant value x can be converted to type T in any of these cases:
>
>- […]
>
>
>- x is an integer or a slice of bytes or runes and T is a string type.
>
>
>- x is a string and T is a slice of bytes or runes.
>
> This means the compiler, when seeing a conversion, explicitly tests for
these two cases.

How the conversion then actually works, you can see in the compiler explorer
. The compiler emits a call into a
function called `runtime.stringtoslicebyte`.
You can actually find that function in the source code of the runtime

then (the same function also contains other functions implementing similar
conversions).
Really, it just allocates a new `[]byte` of the appropriate size and then
copies over the bytes from the string.

Or, is that an internal level detail that an earlier stage (parsing)
>

You can actually mess with this a little bit to show that it's not done in
the parsing stage, but that the compiler actually knows if a type is a
slice of bytes or not.
Because `byte` is a predeclared identifier, it can be shadowed, like every
other identifier. That is, you can declare your *own* type called `byte`:
https://play.golang.org/p/6vDjw9eWX9s
You can also define your own alias for `uint8` and then convert that:
https://play.golang.org/p/nvseU7ofRru
Or you can do both - first shadown the builtin `byte` alias and *then*
create your own:
https://play.golang.org/p/1Y80stIxa5v
You can also define your own type called `uint8` and then define `byte` as
an alias to that and see that you can no longer convert:
https://play.golang.org/p/xUQrAmMm5Km

All of this shows that the compiler really can't just rely on parsing and
the name. It really needs to have a notion of whether something is a slice
of a specific pre-declared type `uint8`, no matter what it is called in the
source code.

It does that by creating a "virtual" package builtin
 inside the compiler and then synthesizing
type-definitions in that package. The code for that is here

.
But it should be noted that this package doesn't really exist and behaves
significantly different from "normal" packages - not just because it is
implemented entirely in the compiler/runtime, but also because it's
identifier are defined "outside" any package, in the universe block
.

I assume this covers all questions :) Let us know if you have follow-ups :)

Axel

takes care of when the compiler sees that statement?
>
> Thanks,
> Amit
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CANODV3nBYmshDLFwUUdnnyVuvpjhWnwBJOb%3DwrZKEHXmtBgbSg%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfHLRuoQtV3uyvF8vHsJUpL5jEFcBJNs88PicZSs3%2Bhpsw%40mail.gmail.com.

Re: [go-nuts] Understanding how []byte("astring") works

[Roland Müller]

Hello,

Am Mo., 14. Juni 2021 um 03:24 Uhr schrieb Amit Saha :

> Hi - My main motivation to understand this is i always had to google
> this - how to convert a string to a byte slice.
>
> Is []byte a type that has been defined in the compiler?
>
> Or, is that an internal level detail that an earlier stage (parsing)
> takes care of when the compiler sees that statement?
>
> Thanks,
> Amit
>
>
a []byte is a sequence of octets and strings in Go consistent of a bytes.
These bytes represent an sequence of unicode characters according to UTF-8.
One such character consists of either a single or two bytes. ASCII -only
strings than have as many bytes as UTF characters.

In the example I made two loop functions loopStringByBytes(s)
and loopStringByChars(s) and checked them against a ASCII string and a
cyrillic string. You can see that for second string every character
occupies 2 bytes.

https://play.golang.org/p/DDSpiFuR8Lp


BR,
Roland



> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CANODV3nBYmshDLFwUUdnnyVuvpjhWnwBJOb%3DwrZKEHXmtBgbSg%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2B8p0G1_iVNyc4MQ_RDFigDRRNVqGzLX_b0E3YSdPXeYbuWZXw%40mail.gmail.com.

[go-nuts] Understanding how []byte("astring") works

[Amit Saha]

Hi - My main motivation to understand this is i always had to google
this - how to convert a string to a byte slice.

Is []byte a type that has been defined in the compiler?

Or, is that an internal level detail that an earlier stage (parsing)
takes care of when the compiler sees that statement?

Thanks,
Amit

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CANODV3nBYmshDLFwUUdnnyVuvpjhWnwBJOb%3DwrZKEHXmtBgbSg%40mail.gmail.com.