[
https://issues.apache.org/jira/browse/ARROW-17584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599144#comment-17599144
]
Tim Schaub edited comment on ARROW-17584 at 9/1/22 8:51 PM:
------------------------------------------------------------
[~zeroshade] - it looks like things should work with `unsafe.Slice`. I found
that tinygo also does not yet implement `map.LoadAndDelete` for `sync`, but
I've proposed adding that here https://github.com/tinygo-org/tinygo/pull/3118
I started to put together a change that converts the casting functions to use
`unsafe.Slice`, but I'm struggling with what the appropriate len/cap should be.
For example, I was assuming `bytesTOUint64` in arrow/bitutil/bitutil.go might
look something like this:
{code:go}
func bytesToUint64(b []byte) []uint64 {
return unsafe.Slice((*uint64)(unsafe.Pointer(&b[0])),
len(b)/uint64SizeBytes)
}
{code}
But I'm uncertain if that is the correct length/capacity. If you have thoughts
on the correct implementation, I would be happy to put together a pull request
that changes this and the other casting functions.
was (Author: JIRAUSER295102):
[~zeroshade] - it looks like things should work with `unsafe.Slice`. I found
that tinygo also does not yet implement `map.LoadAndDelete` for `sync`, but
I've proposed adding that here https://github.com/tinygo-org/tinygo/pull/3118
I started to put together a change that converts the casting functions to use
`unsafe.Slice`, but I'm struggling with what the appropriate len/cap should be.
For example, I was assuming `bytesTOUint64` in arrow/bitutil/bitutil.go might
look something like this:
{code:go}
func bytesToUint64(b []byte) []uint64 {
h := (*reflect.SliceHeader)(unsafe.Pointer(&b))
return unsafe.Slice((*uint64)(unsafe.Pointer(h.Data)),
len(b)/uint64SizeBytes)
}
{code}
But I'm uncertain if that is the correct length/capacity. If you have thoughts
on the correct implementation, I would be happy to put together a pull request
that changes this and the other casting functions.
> [Go] Unable to build with tinygo
> --------------------------------
>
> Key: ARROW-17584
> URL: https://issues.apache.org/jira/browse/ARROW-17584
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
> Reporter: Tim Schaub
> Priority: Major
>
> I was hoping to use TinyGo to build WASM binaries with Arrow. TinyGo can
> generate builds that are [1% the
> size|https://tinygo.org/getting-started/overview/#:~:text=The%20only%20difference%20here%2C%20is,used%2C%20and%20the%20associated%20runtime.&text=In%20this%20case%20the%20Go,size%20(251k%20before%20stripping)!]
> of those generated with Go (significant for applications hosted on the web).
> Arrow's use of `reflect.SliceHeader` fields limits the portability of the
> code. For example, the `Len` and `Cap` fields are assumed to be `int` here:
> https://github.com/apache/arrow/blob/go/v9.0.0/go/arrow/bitutil/bitutil.go#L158-L159
> Go's [reflect package
> warns|https://github.com/golang/go/blob/go1.19/src/reflect/value.go#L2675-L2685]
> that the SliceHeader "cannot be used safely or portably and its
> representation may change in a later release."
> Attempts to build a WASM binary using the github.com/apache/arrow/go/v10
> module result in failures like this:
> {code}
> tinygo build -tags noasm -o test.wasm ./main.go
> {code}
> {code}
> # github.com/apache/arrow/go/v10/arrow/bitutil
> ../../go/pkg/mod/github.com/apache/arrow/go/v10@v10.0.0-20220831082949-cf27001da088/arrow/bitutil/bitutil.go:158:10:
> invalid operation: h.Len / uint64SizeBytes (mismatched types uintptr and int)
> ../../go/pkg/mod/github.com/apache/arrow/go/v10@v10.0.0-20220831082949-cf27001da088/arrow/bitutil/bitutil.go:159:10:
> invalid operation: h.Cap / uint64SizeBytes (mismatched types uintptr and int)
> {code}
> This happens because TinyGo uses `uintptr` for the corresponding types:
> https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/reflect/value.go#L773-L777
> This feels like an issue with TinyGo, and it has been ticketed there multiple
> times (see https://github.com/tinygo-org/tinygo/issues/1284). They lean on
> the warnings in the Go sources that use of the SliceHeader fields makes code
> unportable and suggest changes to the libraries that do not heed this warning.
> I don't have a suggested fix or alternative for Arrow's use of SliceHeader
> fields, but I'm wondering if there would be willingness on the part of this
> package to make WASM builds work with TinyGo. Perhaps the TinyGo authors
> could also offer suggested changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
