Re: [go-nuts] Race detector plus Cgo via binutils extld on arm64?

[Neil Schellenberger]

I'd swear I checked for existing issues but somehow I missed
https://github.com/golang/go/issues/49102.

That was the culprit.  Sorry for the noise.

I suspect I missed it because it's marked closed.  Hopefully something more
overt will be done to spare future poor souls the grief?

.

On Tue, Nov 23, 2021 at 2:34 AM Neil Schellenberger <
neilschellenber...@gmail.com> wrote:

> Thank you very much, Ian, for your insight.
>
> Unfortunately, after several more hours of fiddling, I still can't
> convince it to work.  I assume it's something to do with our (admittedly
> somewhat complicated) build environment/toolchain rather than a core Go
> problem, but I can't figure out what/how.  (Yocto SDK foisted on us by the
> hardware vendor)
>
> I raised https://github.com/golang/go/issues/49749 in the hope that
> someone smarter than me can provide me with the requisite smack upside the
> head
>
> I still get various undefined tsan symbols.  Presumably those are supposed
> to be resolved by the arm64 runtime?  (If it makes any difference, we're
> cross-compiling on amd64 for arm64 by setting GOARCH, CC, CGO_CFLAGS et
> al.  Unfortunately, it's not entirely trivial to build and run a native
> toolchain and Go distro on the hardware in question.)
>
> I've made a rudimentary stab at hunting through the tmp build space to
> figure it out using `go build -race -work -a`, but it's slow going for a
> beginner  FWIW I don't see any obvious signs of any tsan symbol defs in
> there.  (But of course, that may be entirely expected if the runtime module
> is handled specially/magically  And, also, I may simply have missed
> them.  Or be barking up entirely the wrong tree.  In the wrong forest.)
>
> Regards,
> Neil
>
>
> On Fri, Nov 19, 2021 at 3:12 PM Ian Lance Taylor  wrote:
>
>> On Fri, Nov 19, 2021 at 8:59 AM neilschel...@gmail.com
>>  wrote:
>> >
>> > Can anyone comment on whether arm64 race detection plus Cgo (via gcc
>> and binutils) is known to work or not?
>> >
>> > I'm running in to a bunch of mechanical problems getting a binary
>> linked, let alone running.  Unfortunately, we have unavoidable Cgo
>> dependencies on a h/w support library.  I've gotten to the point where the
>> final link is dying with some DWARF warnings and with some relocation
>> errors relating to tsan symbols.
>> >
>> > I don't want to pour too much more time into it if it is known to be a
>> non-starter
>>
>> As of Go 1.16 the Go race detector is supposed to work on linux-arm64
>> and darwin-arm64.  This should work with cgo as well.
>>
>> To be clear, this means building Go code (that can use cgo) with "go
>> build -race".  Combining the Go race detector with the C race detector
>> (the C compiler -fsanitize=thread option) does not work.
>>
>> Please feel free to report bugs at https://golang.org/issue with a
>> description of how we can reproduce the problem.  Thanks.
>>
>> Ian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAM3gH59Qm76Pp%3D2-gG8PE6_HMZy6b7bWUa4aKRvF4Q6Zy29a6w%40mail.gmail.com.

Re: [go-nuts] Race detector plus Cgo via binutils extld on arm64?

[Neil Schellenberger]

Thank you very much, Ian, for your insight.

Unfortunately, after several more hours of fiddling, I still can't convince
it to work.  I assume it's something to do with our (admittedly somewhat
complicated) build environment/toolchain rather than a core Go problem, but
I can't figure out what/how.  (Yocto SDK foisted on us by the hardware
vendor)

I raised https://github.com/golang/go/issues/49749 in the hope that someone
smarter than me can provide me with the requisite smack upside the head

I still get various undefined tsan symbols.  Presumably those are supposed
to be resolved by the arm64 runtime?  (If it makes any difference, we're
cross-compiling on amd64 for arm64 by setting GOARCH, CC, CGO_CFLAGS et
al.  Unfortunately, it's not entirely trivial to build and run a native
toolchain and Go distro on the hardware in question.)

I've made a rudimentary stab at hunting through the tmp build space to
figure it out using `go build -race -work -a`, but it's slow going for a
beginner  FWIW I don't see any obvious signs of any tsan symbol defs in
there.  (But of course, that may be entirely expected if the runtime module
is handled specially/magically  And, also, I may simply have missed
them.  Or be barking up entirely the wrong tree.  In the wrong forest.)

Regards,
Neil


On Fri, Nov 19, 2021 at 3:12 PM Ian Lance Taylor  wrote:

> On Fri, Nov 19, 2021 at 8:59 AM neilschel...@gmail.com
>  wrote:
> >
> > Can anyone comment on whether arm64 race detection plus Cgo (via gcc and
> binutils) is known to work or not?
> >
> > I'm running in to a bunch of mechanical problems getting a binary
> linked, let alone running.  Unfortunately, we have unavoidable Cgo
> dependencies on a h/w support library.  I've gotten to the point where the
> final link is dying with some DWARF warnings and with some relocation
> errors relating to tsan symbols.
> >
> > I don't want to pour too much more time into it if it is known to be a
> non-starter
>
> As of Go 1.16 the Go race detector is supposed to work on linux-arm64
> and darwin-arm64.  This should work with cgo as well.
>
> To be clear, this means building Go code (that can use cgo) with "go
> build -race".  Combining the Go race detector with the C race detector
> (the C compiler -fsanitize=thread option) does not work.
>
> Please feel free to report bugs at https://golang.org/issue with a
> description of how we can reproduce the problem.  Thanks.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAM3gH5-X9X8bTHk8t%3DpyY4p-VcGavWAVv0g7emsAf2HXScCDaw%40mail.gmail.com.

Re: [go-nuts] Timer.Reset() (again...)

[Neil Schellenberger]

Thank you very much for confirming that, Ian!

FWIW the scenario is very roughly along the lines of a "best effort 
re-configurable" timeout for a work loop:

timeout := <-timeoutConfigC
timer := time.NewTimer(timeout)
defer timer.Stop()
for {
  select {
case timeout = <-timeoutConfigC:
  timer.Reset(timeout) // if the user's new preference "just misses" 
this interval, oh well, next time then
case <-timer.C:
  handleTimeout()
  timer.Reset(timeout)
case work := <-workC:
  if done := do(work); done {
return
  }
  }
}

The actual use case is a protocol state machine with various user tunable 
intervals.  It's fine if a new value doesn't get "applied" until the "next 
time around".  

Obviously there are plenty of other ways to handle this, but it got me 
wondering if I understood "Timer.Reset()" properly or not.  And/or I may be 
using an anti-pattern.  Do please let me know if I am :-)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/66358963-9cc4-400d-ab8f-ab6dd1ee5184%40googlegroups.com.

[go-nuts] Timer.Reset() (again...)

[Neil Schellenberger]

Hi Folks,

I am almost certainly overthinking this or in some other way "doing it 
wrong"--so please (dis)abuse me.

The Timer.Reset() documentation reads (in part) "Reset should be invoked 
only on stopped or expired timers with drained channels."

Am I correct in understanding that "should" in that sentence means that if 
neither condition holds then calling Reset() nevertheless still results in 
legal, well defined behaviour?  I.e. "should" (lest you e.g. 
unintentionally race) not "must" (lest you e.g. hang).

>From inspection of the implementation it appears that the only real 
consequence of the inherent race is that it might result in either the 
"old" or "new" expiry being delivered on the next receive.  And even if the 
"old" expiry is still pending in the (buffered, size one) channel when the 
runtime attempts to deliver the "new" expiry, the "new" expiry is simply 
dropped rather than causing any type of deadlock or similar grief to the 
runtime (i.e. per sendTime(), literally the same callback as for 
time.Ticker).

I apologise in advance for further belabouring this hoary chestnut ([1][2][3] 
et multa al. e.g. in StackExchange, Reddit etc.) but I've gotten myself 
thoroughly confused as to what is *guaranteed* by the specification.  Taken 
precisely as written, the admonishments are simply cautions--particularly 
when read in conjunction with the Timer.Stop() documentation (which uses 
less proscriptive lanaguage).  But they are so lengthy that they left me 
with the nagging sense that there might be something more subtle and dire 
lurking behind that I was failing to grasp.  (I.e. no matter how carefully 
written the documentation some b*gger--me--will always misunderstand, 
misinterpret, or otherwise misconstrue it.)

Regards,
Neil

[1] https://github.com/golang/go/issues/14383
[2] https://github.com/golang/go/issues/11513
[3] https://groups.google.com/forum/#!topic/golang-dev/c9UUfASVPoU


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b1bbffe7-594d-400d-ac73-686e43548807%40googlegroups.com.

Re: [go-nuts] cgo behaviour change between go1.11.10 and go1.12.5

[Neil Schellenberger]

Sorry for the delay in responding.  Filed as
https://github.com/golang/go/issues/32579.

On Tue, Jun 11, 2019 at 1:21 AM Ian Lance Taylor  wrote:

> On Mon, Jun 10, 2019 at 9:14 PM Neil Schellenberger
>  wrote:
> >
> > When upgrading our build container from 1.11.10 to 1.12.5, some of our
> tests failed in a surprising way.  I've boiled down the issue to this short
> example code fragment.
> >
> > package main
> >
> > // #include 
> > // struct S { unsigned char data[18]; };
> > import "C"
> > import "unsafe"
> >
> > func main() {
> > var array [5]C.struct_S
> > for i := range array {
> > C.memset(unsafe.Pointer(&array[i].data[0]), 0xff, C.sizeof_struct_S)
> > }
> > for i := range array {
> > for j := range array[i].data {
> > if array[i].data[j] != C.uchar(0xff) {
> > panic("oops")
> > }
> > }
> > }
> > }
> >
> > Under 1.11 this runs to completion; under 1.12 it panics.
> >
> > Debugging and instrumenting shows that the problem is the first argument
> to memset is not what one would expect when run under 1.12.  (The address
> of a heap temporary copy rather than the original perhaps?)
> >
> > Strangely, either parenthesising the argument expression to the address
> operand ("&(array[i].data[0])") or removing the final index expression
> ("&array[i].data") "fixes" the problem.
> >
> > I've spelunked through the AST (identical under both); the liveness and
> escape analysis (different); the parse tree (different); and the go asm
> (different).  Frankly, my gc-fu is very weak and I don't really know what
> I'm looking for.
> >
> > I assume that the root cause is that we were violating some sort of
> liveness or escape policy, but I can't figure out what it would be (or why
> minor syntactic changes "fix" things).  I'd be quite curious to know.
> >
> > The workaround in the real code is simple, so this isn't a huge issue,
> just a a bit of a mystery.
>
> It's a bug.  Would you mind opening an issue with this test case at
> https://golang.org/issue?  Thanks.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAM3gH5_3By%2B37NxrvarWqTw3h-aAuC4owyJnELhpE%3DDcQJhMLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] cgo behaviour change between go1.11.10 and go1.12.5

[Neil Schellenberger]

Hi Folks,

When upgrading our build container from 1.11.10 to 1.12.5, some of our 
tests failed in a surprising way.  I've boiled down the issue to this short 
example code fragment.

package main

// #include 
// struct S { unsigned char data[18]; };
import "C"
import "unsafe"

func main() {
var array [5]C.struct_S
for i := range array {
C.memset(unsafe.Pointer(&array[i].data[0]), 0xff, C.sizeof_struct_S)
}
for i := range array {
for j := range array[i].data {
if array[i].data[j] != C.uchar(0xff) {
panic("oops")
}
}
}
}

Under 1.11 this runs to completion; under 1.12 it panics.

Debugging and instrumenting shows that the problem is the first argument to 
memset is not what one would expect when run under 1.12.  (The address of a 
heap temporary copy rather than the original perhaps?)

Strangely, either parenthesising the argument expression to the address 
operand ("&(array[i].data[0])") or removing the final index expression 
("&array[i].data") "fixes" the problem.

I've spelunked through the AST (identical under both); the liveness and 
escape analysis (different); the parse tree (different); and the go asm 
(different).  Frankly, my gc-fu is very weak and I don't really know what 
I'm looking for.

I assume that the root cause is that we were violating some sort of 
liveness or escape policy, but I can't figure out what it would be (or why 
minor syntactic changes "fix" things).  I'd be quite curious to know.

The workaround in the real code is simple, so this isn't a huge issue, just 
a a bit of a mystery.

Regards,
Neil

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5528b76b-c182-4cab-b3a8-ed5216042259%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Build constraints and preceding non-line comments

[Neil Schellenberger]

Hi Folks,

The documentation of build constraints  
says, in part (emphasis mine):

Constraints may appear in any kind of source file (not just Go), but they 
> must appear near the top of the file, *preceded only by blank lines and 
> other line comments*.


The current implementation 

 (and 
presumably all past implementations) explicitly enforces the constraint 
that a preceding general (i.e. non-line) comment is not permitted.

Does anyone remember the rationale for that?  Something to do with avoiding 
undesired coincidental matching of comments in non-Go files?  I've done a 
little forensic digging, but it quickly gets tricky as the code originated 
in cmd/vet.

I ran across this recently when attempting to insert a build constraint 
into some of our existing Go code with a boilerplate corporate copyright 
which started life in C/C++ source files.  It took me quite a while to 
figure out what was going on  :-)

Regards,
Neil

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d4626304-c167-4c5b-a408-1a46a736f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.