Re: [go-nuts] Re: Casting []byte to []uint64

[Bill Katz]

Ah, of course, silly me conflating alignment with allocation size.  Thanks 
for the suggestion.

Best,
Bill

On Monday, February 27, 2017 at 3:35:32 PM UTC-5, Ian Lance Taylor wrote:
>
> On Mon, Feb 27, 2017 at 11:57 AM, Bill Katz  > wrote: 
> > I thought that's handled by the inhdr.Cap % 8 == 0 check.  Isn't 
> inhdr.Data 
> > essentially a pointer into memory with inhdr.Cap showing the size of the 
> > allocated region? 
>
> Yes, but that is not sufficient.  Consider 
> b := make([]byte, 9) 
> u, err := byteToUint64(b[1:]) 
> Here the len and cap of b are 8, but the data field will not be aligned. 
>
> I think your code is safe but there is a simpler way to do it: 
>
> if len(b) % 8 != 0 || cap(b) % 8 != 0 || 
> uintptr(unsafe.Pointer([0])) % 8  != 0 { 
> return errors.New("bad len or cap or alignment") 
> } 
> return (*[1<<27]uint64)(unsafe.Pointer([0]))[:len(b)/8:cap(b)/8] 
>
> Ian 
>
> > On Monday, February 27, 2017 at 2:14:46 PM UTC-5, xingtao zhao wrote: 
> >> 
> >> I think you need to check if inhdr.Data is aligned with 8 bytes as 
> well. 
> >> 
> >> On Sunday, February 26, 2017 at 2:54:00 PM UTC-8, Bill Katz wrote: 
> >>> 
> >>> Hi, 
> >>> 
> >>> We'd like to do a zero-copy conversion of []byte to []uint64 where the 
> >>> underlying memory for the byte slice is properly aligned for N uint64 
> >>> values.  After looking at (6) in the unsafe package documentation, I'm 
> >>> wondering if the code below is valid because I only use 
> reflect.SliceHeader 
> >>> as pointers, and it seems to fit pattern 6. 
> >>> 
> >>> https://play.golang.org/p/FxxKIK08pX 
> >>> 
> >>> package main 
> >>> 
> >>> import ( 
> >>> "fmt" 
> >>> "reflect" 
> >>> "unsafe" 
> >>> ) 
> >>> 
> >>> func byteToUint64(b []byte) (out []uint64, err error) { 
> >>> inhdr := *(*reflect.SliceHeader)(unsafe.Pointer()) 
> >>> if inhdr.Len % 8 != 0 { 
> >>> err = fmt.Errorf("cannot convert non-aligned []byte length (%d) to 
> >>> []uint64", inhdr.Len) 
> >>> return 
> >>> } 
> >>> if inhdr.Cap % 8 != 0 { 
> >>> err = fmt.Errorf("cannot convert non-aligned []byte capacity (%d) to 
> >>> []uint64", inhdr.Cap) 
> >>> return 
> >>> } 
> >>> outhdr := *(*reflect.SliceHeader)(unsafe.Pointer()) 
> >>> outhdr.Len = inhdr.Len / 8 
> >>> outhdr.Cap = inhdr.Cap / 8 
> >>> outhdr.Data = inhdr.Data 
> >>> out = *(*[]uint64)(unsafe.Pointer()) 
> >>> return 
> >>> } 
> >>> 
> >>> func main() { 
> >>> b := make([]byte, 24) 
> >>> u, err := byteToUint64(b) 
> >>> if err != nil { 
> >>> fmt.Println(err) 
> >>> } else { 
> >>> fmt.Printf("b: %v\n", b) 
> >>> fmt.Printf("u: %v\n", u) 
> >>> fmt.Printf("len(u) = %d, cap(u) = %d\n", len(u), cap(u)) 
> >>> u[0] = 15 
> >>> fmt.Printf("Set u[0] to 15\n") 
> >>> fmt.Printf("b: %v\n", b) 
> >>> fmt.Printf("u: %v\n", u) 
> >>> u[2] = 255 
> >>> fmt.Printf("Set u[2] to 255\n") 
> >>> fmt.Printf("b: %v\n", b) 
> >>> fmt.Printf("u: %v\n", u) 
> >>> } 
> >>> } 
> >>> 
> >>> Outputs: 
> >>> 
> >>> b: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 
> >>> u: [0 0 0] 
> >>> len(u) = 3, cap(u) = 3 
> >>> Set u[0] to 15 
> >>> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 
> >>> u: [15 0 0] 
> >>> Set u[2] to 255 
> >>> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 0 0 0 0 0 0 0] 
> >>> u: [15 0 255] 
> >>> 
> >>> 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Casting []byte to []uint64

[Bill Katz]

I thought that's handled by the inhdr.Cap % 8 == 0 check.  Isn't inhdr.Data 
essentially a pointer into memory with inhdr.Cap showing the size of the 
allocated region?

On Monday, February 27, 2017 at 2:14:46 PM UTC-5, xingtao zhao wrote:
>
> I think you need to check if inhdr.Data is aligned with 8 bytes as well.
>
> On Sunday, February 26, 2017 at 2:54:00 PM UTC-8, Bill Katz wrote:
>>
>> Hi,
>>
>> We'd like to do a zero-copy conversion of []byte to []uint64 where the 
>> underlying memory for the byte slice is properly aligned for N uint64 
>> values.  After looking at (6) in the unsafe package documentation 
>> , I'm wondering if the code 
>> below is valid because I only use reflect.SliceHeader as pointers, and it 
>> seems to fit pattern 6.
>>
>> https://play.golang.org/p/FxxKIK08pX
>>
>> package main
>>
>> import (
>> "fmt"
>> "reflect"
>> "unsafe"
>> )
>>
>> func byteToUint64(b []byte) (out []uint64, err error) {
>> inhdr := *(*reflect.SliceHeader)(unsafe.Pointer())
>> if inhdr.Len % 8 != 0 {
>> err = fmt.Errorf("cannot convert non-aligned []byte length (%d) to 
>> []uint64", inhdr.Len)
>> return
>> }
>> if inhdr.Cap % 8 != 0 {
>> err = fmt.Errorf("cannot convert non-aligned []byte capacity (%d) to 
>> []uint64", inhdr.Cap)
>> return
>> }
>> outhdr := *(*reflect.SliceHeader)(unsafe.Pointer())
>> outhdr.Len = inhdr.Len / 8
>> outhdr.Cap = inhdr.Cap / 8
>> outhdr.Data = inhdr.Data
>> out = *(*[]uint64)(unsafe.Pointer())
>> return
>> }
>>
>> func main() {
>> b := make([]byte, 24)
>> u, err := byteToUint64(b)
>> if err != nil {
>> fmt.Println(err)
>> } else {
>> fmt.Printf("b: %v\n", b)
>> fmt.Printf("u: %v\n", u)
>> fmt.Printf("len(u) = %d, cap(u) = %d\n", len(u), cap(u))
>> u[0] = 15
>> fmt.Printf("Set u[0] to 15\n")
>> fmt.Printf("b: %v\n", b)
>> fmt.Printf("u: %v\n", u)
>> u[2] = 255
>> fmt.Printf("Set u[2] to 255\n")
>> fmt.Printf("b: %v\n", b)
>> fmt.Printf("u: %v\n", u)
>> }
>> }
>>
>> Outputs:
>>
>> b: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
>> u: [0 0 0]
>> len(u) = 3, cap(u) = 3
>> Set u[0] to 15
>> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
>> u: [15 0 0]
>> Set u[2] to 255
>> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 0 0 0 0 0 0 0]
>> u: [15 0 255]
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Casting []byte to []uint64

[xingtao zhao]

I think you need to check if inhdr.Data is aligned with 8 bytes as well.

On Sunday, February 26, 2017 at 2:54:00 PM UTC-8, Bill Katz wrote:
>
> Hi,
>
> We'd like to do a zero-copy conversion of []byte to []uint64 where the 
> underlying memory for the byte slice is properly aligned for N uint64 
> values.  After looking at (6) in the unsafe package documentation 
> , I'm wondering if the code below 
> is valid because I only use reflect.SliceHeader as pointers, and it seems 
> to fit pattern 6.
>
> https://play.golang.org/p/FxxKIK08pX
>
> package main
>
> import (
> "fmt"
> "reflect"
> "unsafe"
> )
>
> func byteToUint64(b []byte) (out []uint64, err error) {
> inhdr := *(*reflect.SliceHeader)(unsafe.Pointer())
> if inhdr.Len % 8 != 0 {
> err = fmt.Errorf("cannot convert non-aligned []byte length (%d) to 
> []uint64", inhdr.Len)
> return
> }
> if inhdr.Cap % 8 != 0 {
> err = fmt.Errorf("cannot convert non-aligned []byte capacity (%d) to 
> []uint64", inhdr.Cap)
> return
> }
> outhdr := *(*reflect.SliceHeader)(unsafe.Pointer())
> outhdr.Len = inhdr.Len / 8
> outhdr.Cap = inhdr.Cap / 8
> outhdr.Data = inhdr.Data
> out = *(*[]uint64)(unsafe.Pointer())
> return
> }
>
> func main() {
> b := make([]byte, 24)
> u, err := byteToUint64(b)
> if err != nil {
> fmt.Println(err)
> } else {
> fmt.Printf("b: %v\n", b)
> fmt.Printf("u: %v\n", u)
> fmt.Printf("len(u) = %d, cap(u) = %d\n", len(u), cap(u))
> u[0] = 15
> fmt.Printf("Set u[0] to 15\n")
> fmt.Printf("b: %v\n", b)
> fmt.Printf("u: %v\n", u)
> u[2] = 255
> fmt.Printf("Set u[2] to 255\n")
> fmt.Printf("b: %v\n", b)
> fmt.Printf("u: %v\n", u)
> }
> }
>
> Outputs:
>
> b: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
> u: [0 0 0]
> len(u) = 3, cap(u) = 3
> Set u[0] to 15
> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
> u: [15 0 0]
> Set u[2] to 255
> b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 0 0 0 0 0 0 0]
> u: [15 0 255]
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.