[go-nuts] Re: 1.13.4: go mod: weird permissions in pkg/mod after go get

['Bryan Mills' via golang-nuts]

The go/loader issue looks like https://golang.org/issue/34860 combined with 
some other bug in one of the go/loader error-handling paths.
(Note that in general we recommend golang.org/x/tools/go/packages 
 for Go tools that 
support module mode.)

What is the underlying problem you were trying to solve when you ran into 
trouble with permissions?


On Saturday, November 23, 2019 at 4:40:20 AM UTC-5, clement auger wrote:
>
> in regard of Sean comment and FTR, 
>
> Now using version go1.13.4
> [clementauger@Host-001 ~] $ ll gow/
> bin/ pkg/ src/ 
> [clementauger@Host-001 ~] $ ll gow/
> bin/ pkg/ src/ 
> [clementauger@Host-001 ~] $ ll gow/pkg/mod/
> cache/  github.com/ golang.org/ gopkg.in/   
> [clementauger@Host-001 ~] $ ll gow/pkg/mod/github.com/apex/
> total 4
> dr-xr-xr-x 5 clementauger clementauger 4096 15 oct.  21:00 log@v1.1.1
> [clementauger@Host-001 ~] $ ll gow/pkg/mod/github.com/apex/*/*
> -r--r--r--  1 clementauger clementauger  794 15 oct.  21:00 gow/pkg/mod/
> github.com/apex/log@v1.1.1/default.go
> -r--r--r--  1 clementauger clementauger  430 15 oct.  21:00 gow/pkg/mod/
> github.com/apex/log@v1.1.1/doc.go
>
> $ ll gow/pkg/mod/github.com/hashicorp/
> total 8
> dr-x-- 2 clementauger clementauger 4096 22 nov.  20:35 errwrap@v1.0.0
> dr-x-- 2 clementauger clementauger 4096 22 nov.  20:35 
> go-multierror@v1.0.0
>
> $ ll gow/pkg/mod/github.com/hashicorp/*/*
> -r--r--r-- 1 clementauger clementauger  4377 22 nov.  20:35 gow/pkg/mod/
> github.com/hashicorp/errwrap@v1.0.0/errwrap.go
> -r--r--r-- 1 clementauger clementauger  1292 22 nov.  20:35 gow/pkg/mod/
> github.com/hashicorp/errwrap@v1.0.0/errwrap_test.go
> -r--r--r-- 1 clementauger clementauger36 22 nov.  20:35 gow/pkg/mod/
> github.com/hashicorp/errwrap@v1.0.0/go.mod
> -r--r--r-- 1 clementauger clementauger 15977 22 nov.  20:35 gow/pkg/mod/
> github.com/hashicorp/errwrap@v1.0.0/LICENSE
>
>
>
>
> Le vendredi 22 novembre 2019 20:40:56 UTC+1, clement auger a écrit :
>>
>> hi,
>>
>> I have some weird permissions set on my module files after i run go get 
>> on my project.
>>
>> please check my session
>>
>> [clementauger@Host-001 jenjen-stream] $ go version
>> go version go1.13.4 linux/amd64
>> [clementauger@Host-001 jenjen-stream] $ go env
>> GO111MODULE=""
>> GOARCH="amd64"
>> GOBIN=""
>> GOCACHE="/home/clementauger/.cache/go-build"
>> GOENV="/home/clementauger/.config/go/env"
>> GOEXE=""
>> GOFLAGS=""
>> GOHOSTARCH="amd64"
>> GOHOSTOS="linux"
>> GONOPROXY=""
>> GONOSUMDB=""
>> GOOS="linux"
>> GOPATH="/home/clementauger/gow"
>> GOPRIVATE=""
>> GOPROXY="https://proxy.golang.org,direct;
>> GOROOT="/home/clementauger/.gvm/gos/go1.13.4"
>> GOSUMDB="sum.golang.org"
>> GOTMPDIR=""
>> GOTOOLDIR="/home/clementauger/.gvm/gos/go1.13.4/pkg/tool/linux_amd64"
>> GCCGO="gccgo"
>> AR="ar"
>> CC="gcc"
>> CXX="g++"
>> CGO_ENABLED="1"
>> GOMOD="/home/clementauger/gow/src/
>> github.com/clementauger/jenjen-stream/go.mod"
>> CGO_CFLAGS="-g -O2"
>> CGO_CPPFLAGS=""
>> CGO_CXXFLAGS="-g -O2"
>> CGO_FFLAGS="-g -O2"
>> CGO_LDFLAGS="-g -O2"
>> PKG_CONFIG="pkg-config"
>> GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 
>> -fdebug-prefix-map=/tmp/go-build597193614=/tmp/go-build 
>> -gno-record-gcc-switches"
>> [clementauger@Host-001 jenjen-stream] $ sudo rm -fr /home/clementauger/
>> gow/pkg/mod/github.com/hashicorp/go-multierror@v1.0.0/
>> [clementauger@Host-001 jenjen-stream] $ sudo rm -fr /home/clementauger/
>> gow/pkg/mod/github.com/hashicorp/errwrap@v1.0.0/
>> [clementauger@Host-001 jenjen-stream] $ sudo rm -fr /home/clementauger/
>> gow/pkg/mod/github.com/hashicorp/
>> [clementauger@Host-001 jenjen-stream] $ cat go.mod 
>> module github.com/clementauger/jenjen-stream
>>
>> go 1.13
>>
>> require github.com/hashicorp/go-multierror v1.0.0
>> [clementauger@Host-001 jenjen-stream] $ go get ./...
>> go: extracting github.com/hashicorp/go-multierror v1.0.0
>> go: extracting github.com/hashicorp/errwrap v1.0.0
>> [clementauger@Host-001 jenjen-stream] $ ll /home/clementauger/gow/pkg/mod
>> /github.com/hashicorp/
>> total 8
>> dr-x-- 2 clementauger clementauger 4096 22 nov.  20:35 errwrap@v1.0.0
>> dr-x-- 2 clementauger clementauger 4096 22 nov.  20:35 go-
>> multierror@v1.0.0
>>
>> It also triggers a bug when i try to load that package using x/go/loader
>>
>> $ jenjen -template=github.com/clementauger/jenjen-stream -   "T => 
>> string, StreamT => StreamString, stream => streamString"
>> /home/clementauger/gow/pkg/mod/github.com/hashicorp/go-multierror@v1.0.0/
>> prefix.go:6:2: could not import github.com/hashicorp/errwrap (go/build: 
>> importGo github.com/hashicorp/errwrap: exit status 1
>> error writing go.mod: open /home/clementauger/gow/pkg/mod/github.com/
>> hashicorp/go-multierror@v1.0.0/go.mod298498081.tmp: permission denied
>>
>> )
>> 2019/11/22 20:37:34 couldn't load packages due to errors: 
>> github.com/hashicorp/go-multierror
>> panic: runtime error: invalid memory address or nil pointer 

Re: [go-nuts] Re: Struggling with go mod, micro-repos and private forks

['Bryan Mills' via golang-nuts]

On Friday, October 25, 2019 at 10:19:03 AM UTC-4, Shaun Crampton wrote:
>
>
>
> We have https://golang.org/issue/26232 open for 2FA workflows in general.
>>
>> In the meantime, you may need to configure a Personal Access Token 
>> 
>>  to 
>> get HTTPS to work.
>>
>
>
> We use 2FA for accessing the github UI but we tend to use SSH for push and 
> pull.  Would be easier if it tried ssh rather than bailing out.
>

 Ah, that one is https://golang.org/issue/26134.


>>>- Another team member said they tried using 
>>>v0.0.0-0001010100- as version but it got replaced.
>>>
>>> That probably means that you have a higher version requirement from some 
>> other dependency. (The `go` command automatically updates the `go.mod` file 
>> to maintain consistency.)
>>
>>>
>>>
> But then we go round in circles, right?  The start of this discussion was 
> that I wanted a "nil" version to put in my require statement so that I 
> could replace it with exactly the commit that I need using a "replace".  
> Now, if the tool helpfully updates the require then that defeats the point 
> of having a "nil" version in the first place.
>

If you don't care what version you're replacing, use a `replace` directive 
without a version and don't worry about the version that the `go` command 
adds in the `require` directive. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/009b0f6d-4925-4e2c-9b50-c3efdd981198%40googlegroups.com.

[go-nuts] Re: Struggling with go mod, micro-repos and private forks

['Bryan Mills' via golang-nuts]

Thanks for the feedback.

The good news is that we're aware of (and planning to address) most of 
these pain points; the bad news is that we haven't been able to get to most 
of them yet.

Detailed responses inline.

On Tuesday, October 22, 2019 at 6:42:05 AM UTC-4, Shaun Crampton wrote:
>
> Hi All,
>
> My team own a large product that sprawls across a lot of repos.  We also 
> manage a private, commercial fork and we occasionally do new development in 
> private and then merge across to open source.  While a monorepo might have 
> been easier to manage, we're stuck with what we've got.  We've recently 
> moved to go mod and it just seems like we're constantly fighting the tool.  
> I'm hoping you can either suggest some good workflows or maybe improve the 
> tool!
>
> I think the biggest problem we have is when working with private repos.  
>  What I want to express to the tool is 
>
> My module requires commit abcd1234 (or version v1.2.3) of dependency x/y/z
>
> Look for any instances of dependency x/y/z in git repo 
> g...@github.com:ourfork instead of the default.
>
>  
I believe this is https://golang.org/issue/28176.

It's an intriguing idea, but problematic if the dependencies are specified 
as pseudo-versions, because the commit hashes in a fork in general will not 
match the hashes in the upstream module. We're still exploring alternatives 
for this use-case.


However, what I can express to the tool is
>
> My module requires version ??? of dependency x/y/z
>
> Replace  x/y/z @ version ??? with  @ abcd1234
>
>  
>
> This throws up a couple of problems:
>
>- What should version ??? be?  It's only there to be replaced, which 
>seems like a bit of a smell.  
>   - If I set it to the commit ID it gets resolved and I have to 
>   change three places in the file each time I move the pin.
>   - If I set it to a particular version, that seems misleading.  
>   - I guess I can set it to v0.0.0, but again that seems misleading.
>
>
Currently it should be `v0.0.0-0001010100-`.
We're also rethinking this behavior for `replace` in general; 
see https://golang.org/issue/33370.



>- There's nowhere to specify the details of the repo (e.g. 
>connection/auth type), all that has to magically work according to my git 
>settings and the defaults aren't great for private repos (which we access 
>over ssh).
>
> See https://golang.org/cmd/go/#hdr-Remote_import_paths.

In general we expect that either you have an HTTPS server to locate the 
repo for a given import path (credentials can be stored in a `.netrc` file; 
see also https://golang.org/issue/26232), or you include an explicit VCS 
extension somewhere in the import path (and have corresponding credentials 
configured per-user in your VCS).


We also just ran into the new GOPRIVATE env var and had to update all our 
> builds to use that.  Couldn't that just fall back to the private behaviour 
> if it gets a cache miss, it seems over-engineered to require an explicit 
> exception!
>

The checksum database and proxy fallback are designed to provide integrity 
by default.

Without making the opt-out explicit, an origin could serve one set of 
sources (and checksum) to you, and another version to someone else, and 
simply refuse to serve anything at all in response to queries from the 
checksum database (so that the entry would always be missing), and no one 
would be the wiser.


Another issue we've had is making a build mode where we can build against 
> local copies of the dependencies for quick development.  We can add replace 
> directives to point to local directories, which is part of the puzzle, but 
> it's hard to do that "just for one build".  Not sure what we're looking for 
> here; maybe an optional go.mod override file that can be passed in for just 
> one build?
>

You mean like https://golang.org/issue/34506?  

 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/68003fda-f411-4e5f-9657-15b774c176d0%40googlegroups.com.

Re: [go-nuts] how to pass c array to golang efficiently

['Bryan Mills' via golang-nuts]

On Wednesday, August 7, 2019 at 4:21:03 AM UTC-4, Jan Mercl wrote:
>
> On Wed, Aug 7, 2019 at 3:17 AM hui zhang > 
> wrote: 
>
> > //export Send 
> > func Send(confid string, len int, pcm *C.short) { 
> >//put c.short array to int16 slice/array efficiently , how ? 
> >// memcopy ? 
> > } 
>
> s := (*[^uint(0) >> 1]int16)(unsafe.Pointer(pcm))[:len] // s is 
> []int16, len(s) = len 
>

That constant doesn't always work. 
(See https://golang.org/issue/13656#issuecomment-165858089.)

If you have a static upper bound on the size of the array, then it's fine 
to use whatever that bound is, but if you don't then you generally need a 
more dynamic calculation (like the one 
in https://golang.org/issue/13656#issuecomment-303216308).

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/10db45f8-840e-4fc4-829b-82a7844e3beb%40googlegroups.com.

[go-nuts] Re: Prevent a package from being updated using go modules

['Bryan Mills' via golang-nuts]

This sort of use-case is pretty much exactly what the exclude directive is 
for.
(See https://tip.golang.org/cmd/go/#hdr-The_go_mod_file.)

In your go.mod file, add a directive like:

exclude github.com/vendor/package v1.3.0

In the meantime, send a PR or file an issue with github.com/vendor/package to 
fix the bug.

Once it's fixed, you can run go get github.com/vendor/package@$COMMIT at 
whatever commit fixed the bug in order to upgrade past it; then you can 
remove the exclude directive.


On Monday, July 15, 2019 at 2:14:22 PM UTC-4, Pantelis Sampaziotis wrote:
>
> Hi,
>
> I would like to ask if there is a functionality similar to apt-mark hold 
> (which prevents package from being automatically installed, upgraded or 
> removed) in go modules.
>
> The case I have is that after updating a package from 1.2.2 to 1.3.0, a 
> bug was introduced which can break the app in some cases edge cases (when 
> parsing specific json responses) on runtime.
>
> I want to lock down the version to 1.2.2 and make sure that this package 
> is not updated when someone runs go get -u until the bug is fixed.
>
> It seems the replace directive 
> https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive
>  provides 
> similar functionality:
>
> replace github.com/vendor/package => github.com/vendor/package v1.2.2
>
> Is this the correct way? Is there any other solution?
>
> thank you
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/205161bf-1742-4c7f-a5c2-f2935d533b86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] The "leave "if err != nil" alone?" anti-proposal

['Bryan Mills' via golang-nuts]

On Monday, July 1, 2019 at 3:12:24 AM UTC-4, Ian Lance Taylor wrote:
>
> Think about how you would write a basic Go construct like 
> https://godoc.org/golang.org/x/sync/errgroup if errors are handled via 
> exceptions.  I'm not saying you can't do it--of course you can do it. 
> But it seems to me that it would require a bunch of awkward 
> boilerplate that would be easy to get wrong. 
>

That reminds me, I need to dust off CL 134395  
(the change to make `errgroup` work with things like `(*testing.T).Fatal)`).

As it turns out, we already have to deal with many of the same complexities 
that exceptions introduce, because `panic` is defined as a control-flow 
mechanism rather than an unambiguous programmer error, and is used as such 
in the `testing` and `fmt` packages (see also #28150 
).

It does indeed require a bunch of awkward boilerplate that is easy to get 
wrong. (See the `doubleDeferSandwich` function in that CL in particular.) 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/08bc1368-0d17-4cf7-a98c-21024e146183%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go version go1.12 and cannot find module providing package

['Bryan Mills' via golang-nuts]

The highest release tag on that module is v0.0.2, and that version indeed does 
not provide the clis package.

If you want a commit that is more recent than the latest release, you need to 
request that commit to 'go get' explicitly.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4933c5b1-8f36-4932-8a63-c4ee372e77b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to properly release a new project as a V2 module

['Bryan Mills' via golang-nuts]

The /v2 suffix is not just for within your code: that's how you need to address 
the module (and import its packages) everywhere.

The contents of your go.mod file suggest that your calling code, or perhaps the 
`go get` command you used to add the module, is missing that suffix.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: What is the future of go mod vendoring?

['Bryan Mills' via golang-nuts]

Vendoring support in 1.13 is currently under discussion 
at https://golang.org/issue/30240.

TL;DR: I'm hoping to better integrate it with the module workflow. (It's 
not going away in the foreseeable future.)


On Friday, February 15, 2019 at 5:05:14 AM UTC-5, Amnon Baron Cohen wrote:
>
>
> https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away
>
> On Thursday, 14 February 2019 02:26:35 UTC, Paul A. Fortin wrote:
>>
>> I have heard that the vendir dirctory is here to stay and also that it is 
>> going away - can someone from the goteam give us a idea of the future of 
>> that feature?
>>
>> Thanks
>>
>> p4tin
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go modules do not respect vendor folder of dependencies (is this by design?)

['Bryan Mills' via golang-nuts]

The fact that a module-mode build ignores dependencies vendored into *other* 
modules 
is intentional.
#27227 may change several details of vendoring in module mode, but this is 
not one of them.

If module B vendors its dependencies but does not specify their versions in 
a go.mod file, then you should specify the versions of those dependencies 
that you want in *your* module's go.mod file. (You can set them to 
something consistent using go get -m.)


On Sunday, February 10, 2019 at 12:26:10 PM UTC-5, Ian Davis wrote:
>
>
> On Sun, 10 Feb 2019, at 12:03 AM, se...@pion.ly  wrote:
>
> * I have a project 'A' that is importing 'B'
> * 'B' uses a vendor folder, if you delete the vendor folder the project 
> does not compile.
> * With `GO111MODULE=on` I am unable to project 'A'. It tries to pull the 
> latest of 'B', and everything 'B' depends on (instead of using 'B's vendor 
> folder)
>
> Does anyone have suggestions on what I should do here? I don't control 
> project 'B' only 'A'
>
>
>
> This sounds like issue https://github.com/golang/go/issues/27227 which is 
> still under discussion
>
> Ian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Existing code for order-preserving concurrent work queue?

['Bryan Mills' via golang-nuts]

The code to sequence the results using a channel is not much more verbose.

That not only avoids the library dependency, but also makes the peak memory 
consumption for the results O(runtime.NumCPU()), instead of O(N) with the 
number of tasks, and allows the output to be streamed instead of buffered 
to a slice.

https://play.golang.org/p/zkBjxlcvESe

(In a more realistic situation I would probably use an errgroup.Group 
instead of a raw `go` statement, but the overall structure remains the 
same.)


On Monday, January 28, 2019 at 3:49:34 AM UTC-5, rog wrote:
>
> FWIW I wrote *https://godoc.org/github.com/juju/utils/parallel#Run 
> * for exactly this 
> kind of use case. It doesn't explicitly mention the values in the API 
> because it's easier (and more type safe) to let the body of the closure 
> write directly to a result slice.
>
> Example: https://play.golang.org/p/ffj_Eb_0B2q
>
>
>
> On Fri, 25 Jan 2019, 10:48 pm  wrote:
>
>> For what it's worth
>>   http://www.golangpatterns.info/concurrency/parallel-for-loop
>> implements an order-preserving parallel map, but does not limit the 
>> number of workers.
>>
>> In my case, I want to limit the number of workers because I'm making a 
>> lot of system calls and don't want to overload the kernel. runtime.NumCPU() 
>> seems like a reasonable limit.
>>
>>
>>
>> On Friday, January 25, 2019 at 8:04:31 PM UTC+1, twp...@gmail.com wrote:
>>>
>>> Hi,
>>>
>>> I have a number of slow tasks that I want to run concurrently across 
>>> runtime.NumCPU() workers in a single process. The tasks have a specific 
>>> input order, but they are completely independent of each other and can 
>>> execute in any order. I would like to print the output of each task in the 
>>> same order as the input order of tasks.
>>>
>>> This can be implemented by including each task's index in the input 
>>> order as it is distributed via a channel to the workers, and the final 
>>> collection of results assembled using these task indexes before the results 
>>> are printed.
>>>
>>> Assumptions:
>>> - Small number of tasks (~10,000 max), i.e. this easily fits in memory.
>>> - Single Go process, i.e. I don't want/need a distributed system.
>>>
>>> This feels like it should be common problem and there's probably either 
>>> a library or a standard Go pattern out there which can do it. My web search 
>>> skills didn't find such a library though. Do you know of one?
>>>
>>> Cheers,
>>> Tom
>>>
>>>
>>> Background info to avoid the XY problem : this 
>>> is to make chezmoi  run faster. I 
>>> want to run the doctor checks 
>>> 
>>>  
>>> (basically os.Exec'ing a whole load of binaries to get their versions) 
>>> concurrently in the short term. In the long term I want to make chezmoi's 
>>> apply concurrent, so it runs faster too. In the first case, the order 
>>> requirement is because I want all users to see the output in the same order 
>>> so that it's easy to compare. In the second case, the order requirement 
>>> comes because I need to ensure that parent directories are in the correct 
>>> state before checking their children.
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: invalid recursive type alias

['Bryan Mills' via golang-nuts]

The compiler is behaving as designed in both cases.

See https://golang.org/issue/25187 (and the 
associated https://golang.org/issue/25141).


On Saturday, December 8, 2018 at 12:33:48 PM UTC-5, Jan Mercl wrote:
>
> This code compiles fine
>
> package main
> 
> type node struct {
> next *node
> }
> 
> func main() {}
>
> (https://play.golang.org/p/ZYg0EciQnOQ)
>
> This code does not
>
> package main
> 
> type node = struct {
> next *node
> }
> 
> func main() {}
>
> (https://play.golang.org/p/gWWX8ngPsS6)
>
> The error is
>
> prog.go:3:6: invalid recursive type alias node
> prog.go:3:6: node uses 
> prog.go:3:13:  uses node
>
> The specification remains silent about what is considered "invalid 
> recursive type alias" so I'm not sure, but it seems to me the first and 
> second programs should both compile fine. In both cases, so to say "the 
> pointer breaks the cycle" while type checking. But maybe/probably I'm 
> mistaken.
>
> Can anybody enlighten me please and explain if the compiler is right or if 
> it's a bug?
>
> Thanks in advance.
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] convert *byte to []byte

['Bryan Mills' via golang-nuts]

On Monday, December 3, 2018 at 12:02:07 AM UTC-5, Ian Lance Taylor wrote:
>
> On Sat, Dec 1, 2018 at 9:39 AM > wrote: 
> > 
> > I am using swig wrap a c++ module , the generated go code is like this: 
> > 
> > type  MediaFrame interface { 
> >  GetLength()  uint 
> >  GetData()  (*byte) 
> > } 
> > 
> > I want to convert the *byte  to []byte,  How to do this? 
>
> One approach is 
>
> s := 
> (*[1<<30]byte)(unsafe.Pointer(mf.GetData())[:mf.GetLength():mf.GetLength()] 
>
> Ian 
>

If you need to handle slices larger than 2³⁰ bytes, a more robust version 
of this same approach is described 
in https://golang.org/issue/13656#issuecomment-303216308.
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] New Modules and git clones

['Bryan Mills' via golang-nuts]

On Wednesday, November 14, 2018 at 2:26:07 PM UTC-5, Paul Jolly wrote:
>
> > > Can you point us at the documentation you're referring to here, 
> please? 
> > 
> >  go help go.mod 
>
> Thanks - it's honestly the first time I've a) read that help document 
> or b) seen the dev version format. I also can't find any tests 
> covering its usage. 
>
> Please can you raise an issue about the use of dev in this context, 
> via https://github.com/golang/go/issues/new? It's entirely possible I 
> am/we are missing something here, but no harm in checking. 
>

The `dev` in that documentation is intended to be a branch name. If that 
module doesn't actually *have* a branch named `dev`, it won't work.


For now, using any version < v2 should work, just so long as it is a 
> valid semver version that therefore doesn't require any further 
> resolution. e.g. v0.0.0 should be fine. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Mini library for structured concurrency.

['Bryan Mills' via golang-nuts]

(Hi, errgroup author here. Comments inline below.)

On Thursday, October 25, 2018 at 7:40:43 AM UTC-4, Andrew Chambers wrote:
>
> @Sebastien
>
> The biggest difference is my package makes guarantees that when a bundle 
> is garbage collected the context and thus child goroutines
> are cancelled. This lets you be more creative having shared ownership of 
> groups of goroutines, e.g.  a collection of goroutines producing a lazy 
> infinite series on a channel, which would be safely closed once all 
> references to the bundle are gone.
>

I don't think I understand your use-case here. If you're producing an 
infinite series on a channel, what prevents the bundle from being collected 
while some goroutine is still reading on that channel? It seems like you 
always need some higher-level structure to own both the bundle and the 
communication mechanism, and at that point you can toss a `Close` method on 
that structure and not need to worry about finalizers.

On the other hand, relying on finalizers to clean up goroutines is itself 
quite risky.
Per the documentation :

   - “If a cyclic structure includes a block with a finalizer, that cycle 
   is not guaranteed to be garbage collected and the finalizer is not 
   guaranteed to run, because there is no ordering that respects the 
   dependencies.”
   - “The finalizer is scheduled to run at some arbitrary time after the 
   program can no longer reach the object to which obj points. There is no 
   guarantee that finalizers will run before a program exits […].”

So if the thing you're storing the bundle in participates in a cycle, the 
goroutines might never be cancelled; if the goroutines in your bundle are 
actively performing work, you might waste an arbitrary amount of CPU before 
they finally stop.

Moreover, if you don't have some explicit point of cancellation, you also 
don't have an explicit point at which you can block for the goroutines to 
return. So how will you test for goroutine leaks? With explicit 
cancellation it's easy: you cancel the bundle and wait for everything to 
return, and if one of the goroutines leaks or deadlocks then your test will 
hang — and you can send it SIGQUIT and look at the goroutine stacks to 
figure out what went wrong. In contrast, with the finalizer approach you 
can't distinguish between a goroutine leak and the collector just taking 
its time.

(See also the first section of my 2018 GopherCon talk, *Rethinking 
Classical Concurrency Patterns* (slides) 
 
(video) .)


errgroup seem to have a 'heavier' api, focused on aggregating errors, 
>

If you aren't collecting errors, and *are* using synchronous functions to 
check for goroutine leaks, then the bundling doesn't provide much benefit 
over using context.WithCancel and a sync.WaitGroup separately: it drops the 
paired Add/Done boilerplate from the WaitGroup, and that's about it. 
`errgroup` has a heavier API because a thinner API doesn't provide enough 
benefit to justify another layer of abstraction.

(See also chapter 4 of John Ousterhout's *A Philosophy of Software Design*.)

 

> It also seems strange to me that the 'group' function in errgroup does not 
> accept a context which is idiomatic go. Being biased, I prefer my own
> API, but actually, use whichever you prefer :) 
>

In `errgroup`, the `ctx context.Context` parameter is omitted to reduce 
boilerplate: it is always the same for every function, so the caller only 
needs to obtain it once, and there is no need to repeat the same 
boilerplate at every call. Furthermore, it is sometimes useful to be able 
to make additional modifications (such as an additional cancellation layer 
or a watchdog timeout) for a subset of the goroutines in the group. So the 
explicit parameter would impose a cost (boilerplate) and remove a benefit 
(injecting finer-grained cancellation) while providing relatively little 
benefit of its own (a slightly lower likelihood of using the wrong context, 
which is already being reduced somewhat in https://golang.org/cl/134395).

Plus, omitting the context parameter makes the zero `errgroup.Group` 
somewhat useful for aggregating errors without cancellation. (A context 
parameter passed in by a Group that was constructed without one would be 
misleading at best: it would suggest propagation of values when it only 
really provides cancellation. See also #28342 
.)

If the language didn't require so much boilerplate for each function 
parameter (see #21498 ), the cost/benefit 
balance might swing back in favor of the explicit parameter.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more 

Re: [go-nuts] efficient way to csv.Reader() from a chan of string

['Bryan Mills' via golang-nuts]

You don't need the io.Pipe: it's fairly trivial to implement io.Reader in 
terms of a channel of strings and a strings.Reader.

You can even implement io.WriterTo efficiently that way: 
https://play.golang.org/p/PxIEQYUoC50

(In theory that can make the whole pipeline zero-copy, but in practice I 
doubt that csv.Reader is that specialized.)


On Sunday, September 16, 2018 at 4:07:35 AM UTC-4, Sherif Eldeeb wrote:
>
> Even though Yannic's answer was really useful and appreciated, I ended up 
> using io.Pipe() which looks simpler and more efficient (explained below):
>
> 
> rp, wp := io.Pipe()
> go func() {
> defer wp.Close()
> for i := range feederChan {
> fmt.Fprintln(wp, i)
> }
> }()
> r := csv.NewReader(rp)
> for { // keep reading
> a, err := r.Read()
> if err == io.EOF {
> break
> }
> // do stuff with 'a'
> // ...
> } 
> 
>
> The io.Pipe() is synchronous, and should be fairly efficient: it pipes 
> data from writer to a reader; I fed the csv.NewReader() the reader part, 
> and created a goroutine that drains the chan writing to the writer part.
>
> Thanks a lot.
>
> On Sun, Sep 16, 2018 at 1:25 AM Yannic Bonenberger <
> yan...@yannic-bonenberger.com > wrote:
>
>> You can use a bytes.Buffer to create a single csv.Reader and write the 
>> log lines into that buffer.
>> The code will look roughly like this 
>> https://play.golang.org/p/gbCPwSsx5gy .
>> However, depending on the implementation of strings.Reader and the level 
>> of optimization the Go compiler does, your approach of creating a new 
>> csv.Reader for every iteration might actually be faster and consume less 
>> memory than my code.
>>
>>
>> Greetings all,
>> I have a channel of 'string', where each item is a single CSV log line 
>> that I want to convert to columns using "encoding/csv" 
>> Currently, I am creating a new csv.Reader at each iteration for each 
>> item, which is much more work than it needs to be.
>>
>> 
>> for i := range feederChan {
>> r := csv.NewReader(strings.NewReader(i))
>> a := r.Read()
>> // Do stuff with a
>> // ...
>> }
>> 
>>
>> I would really appreciate sharing with me if there's a way to iterate 
>> through a 'chan string' using 'csv.Reader()' without the need to create a 
>> new Reader for each item.
>> Thanks.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why can't we use unicode? [Go2 Generics]

['Bryan Mills' via golang-nuts]

Coincidentally, I contribute to a ChromeOS extension that adds XCompose 
support,¹
and I use a similar utility on Windows.²
(I'm not sure what's available on macOS.)

¹ https://github.com/bcmills/extra-keyboards-for-chrome-os/tree/custom-sequences
² https://github.com/samhocevar/wincompose

On Friday, September 7, 2018 at 12:39:38 PM UTC-4, Ian Cottrell wrote:
>
> The same thing works in any X system that supports compose, probably bound 
> to the Shift+AltGr
>
> On Fri, Sep 7, 2018 at 12:29 PM roger peppe  > wrote:
>
>> In acme (and plan 9 generally), there's a nice set of mnemonic
>> abbreviations for unicode characters.
>> It's great, and I miss it in other environments. Alt-<< and Alt->>
>> work really well for « and » for example.
>> Here's the full list:
>> https://github.com/9fans/plan9port/blob/master/lib/keyboard
>>
>> On 7 September 2018 at 16:18, Michael Jones > > wrote:
>> > I brought this up way back in the early days.
>> > There will be an old post.
>> > The fear is mental inertia and muscle memory -- a new-to-beginners 
>> character
>> > set would not "sell".
>> >
>> > An easy compromise is go vet: it can translate between  '>=" to '≥' 
>> rather
>> > easily.
>> >
>> > On Fri, Sep 7, 2018 at 6:17 AM Larry Clapp > > wrote:
>> >>
>> >> Need more shift keys!
>> >>
>> >> I'm pretty sure if I used them every day, I'd learn pretty quickly 
>> that «
>> >> & » are from opt-\ and shift-opt-\, and ‹ & › are from shift-opt-3 & 4.
>> >>
>> >> Windows users ... are on their own.  Find a use for the
>> >> otherwise-poorly-used numeric keypad, maybe.  (Sometimes I wish Macs 
>> could
>> >> tell the difference between 1 and keypad-1, etc, like Windows can.  
>> It'd
>> >> give me a whole new set of hotkeys.  :)
>> >>
>> >> On a (slightly) more serious note -- Would 
>> multiple-punctuation-character
>> >> symbols work?  {<  and >}, or (<  and >) ?  Or <( and )> / <{ & }>.  I 
>> kind
>> >> of like these last two.  Nesting is ... iffy, I guess?
>> >>
>> >> <(<(stuff, <(stuff)>, stuff)>, stuff)>
>> >>
>> >> I'm sure there would be screams, and shouting about Perl, etc.
>> >>
>> >> — Larry
>> >> ^ an M-dash, haha.  Shift-opt-minus.  Easy-peasy.
>> >>
>> >> On Thursday, September 6, 2018 at 8:01:14 PM UTC-4, Axel Wagner wrote:
>> >>>
>> >>> And while we're at it, why "func", instead of the far simpler λ, or
>> >>> "type" instead of τ, or "include", instead of ι, "const" instead of κ 
>> and
>> >>> "war" instead of ω. We can do ρ instead of "range", φ instead of 
>> "for", ν is
>> >>> "new" and μ is "make", obviously. And while we're at it, let's also 
>> use ≥
>> >>> and ≤ and ≠. No * and /, just • and ÷. ¬, ∨, ∧ of course for 
>> booleans. ← and
>> >>> → for channel ops and short variable declaration with ≔.
>> >>>
>> >>> The answer is, that most people don't know how to enter any of these 
>> and
>> >>> the ones that do don't want to be bothered having to change their
>> >>> keyboard-mapping or hammering there num-block for every (or, really, 
>> any)
>> >>> line of code :)
>> >>>
>> >>> On Fri, Sep 7, 2018 at 1:34 AM Wojciech S. Czarnecki <
>> oh...@fairbe.org>
>> >>> wrote:
>> 
>>  I can not understand why, way in the XXIst century, in a language 
>> that
>>  from
>>  the beginning supports for unicode identifiers we are at ascii 
>> charset
>>  overloading bikeshed. Why type `type` or (in other proposal $, or <> 
>> or
>>  [] or
>>  whatever<128) if I might press Super-T and get ʧ. Or press Super-G 
>> and
>>  get ʭ.
>> 
>>  I hear that only gurus will write generic code. Might it be, but
>>  thousands of
>>  rookies should be able to read this generic code before they make 
>> their
>>  first
>>  commit.
>> 
>>  Gurus will know how to map their keyboards. Rookies on their (win)
>>  machines
>>  have circa 1000 glyphs in basic system fonts. (On any linux distro 
>> have
>>  over
>>  3000).
>> 
>>  Why on earth keep on ascii?
>> 
>>  IPA: ʅ ʧ ʭ (0x285, 0x2a7, 0x2ad)
>>  Latin-E: « » ¦
>>  Latin-A: Ħ ŧ Ŧ Ɏ
>>  Latin-B: ǁ ǂ
>> 
>>  --
>>  Wojciech S. Czarnecki
>>   << ^oo^ >> OHIR-RIPE
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google 
>> Groups
>> >> "golang-nuts" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send 
>> an
>> >> email to golang-nuts...@googlegroups.com .
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> >
>> > --
>> > Michael T. Jones
>> > michae...@gmail.com 
>> >
>> > --
>> > You received this message because you are subscribed to the Google 
>> Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an
>> > email to golang-nuts...@googlegroups.com .
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> 

[go-nuts] Re: Go 1.11 module cache not safe for concurrent use?

['Bryan Mills' via golang-nuts]

The module cache and the build cache are separate entities.

The build cache is safe for concurrent use. The module cache is not (yet),¹ 
but you can pre-populate it by running go mod download.

¹ https://golang.org/issue/26794


On Tuesday, August 28, 2018 at 11:01:43 AM UTC-4, Maxim Khitrov wrote:
>
> I'm running into non-deterministic errors when I try to build multiple 
> binaries from the same module with goreleaser in a CI/CD system. Here 
> are some examples: 
>
> go: finding github.com/pkg/browser v0.0.0-20170505125900-c90ca0c84f15 
> go: github.com/pkg/browser@v0.0.0-20170505125900-c90ca0c84f15: git 
> fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in 
> /go/pkg/mod/cache/vcs/8755ee3fe34afbf0e42bd94de4ff233c0a12a60e9ddcc7a72c38f16620d3eebf:
>  
>
> chdir 
> /go/pkg/mod/cache/vcs/8755ee3fe34afbf0e42bd94de4ff233c0a12a60e9ddcc7a72c38f16620d3eebf:
>  
>
> no such file or directory 
>
> go: github.com/davecgh/go-spew@v1.1.1: git remote add origin 
> https://github.com/davecgh/go-spew in 
> /go/pkg/mod/cache/vcs/b9a4b9bbdb4a59723f2348415ad7ffda91568455a1cfd92e97976132bdfbaf57:
>  
>
> exit status 128: 
> fatal: remote origin already exists. 
>
> go: github.com/pmezard/go-difflib@v1.0.0: unknown revision v1.0.0 
>
> go: error loading module requirements 
>
> If I run 'go mod download' before the build, everything works fine, so 
> I believe that this problem is caused by goreleaser starting multiple 
> parallel builds with an empty module cache. I've tested this several 
> times now by removing pkg/mod and GOCACHE directories. 
>
> My question is whether it should be possible to do so or if this is 
> just a known limitation of how the module cache works? The following 
> issue suggests that GOCACHE is safe for concurrent use, but I couldn't 
> find information on the module cache: 
>
> https://github.com/golang/go/issues/26677 
>
> -Max 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Go modules: go get installs binary

['Bryan Mills' via golang-nuts]

Per the documentation 
, “Get resolves and 
adds dependencies to the current development module and then builds and 
installs them.”

If you want it to only update dependencies, use the -m flag; to only update 
and download, use -d.


On Sunday, September 9, 2018 at 8:59:47 AM UTC-4, asv...@gmail.com wrote:
>
> Does it correct that go get (with modules enabled) installs binary. I 
> thought it must only download / update dependencies in modules mode?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Go 1.11 Beta 3 is released

['Bryan Mills' via golang-nuts]

Did you happen to install it using another go1.11 beta build in module mode?

If so, it could have been https://golang.org/issue/26869.


On Thursday, August 9, 2018 at 4:35:31 PM UTC-4, Ken MacDonald wrote:
>
> Hi,
> Just attempted to install 1.11beta3. Using the following instructions, the 
> "go get" portion appears to have succeeded, but "go1.11beta3 download" 
> fails with "bash: go1.11beta3: command not found". This worked fine on my 
> Mac, but trying on a CentOS system now. Suggestions welcome.
>
>
>> If you have Go installed already, the easiest way to try go1.11beta3
>> is by using the go command:
>> $ go get golang.org/dl/go1.11beta3
>> $ go1.11beta3 download
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go could really use a while statement

['Bryan Mills' via golang-nuts]

On Thursday, May 3, 2018 at 4:25:34 AM UTC-4, rog wrote:
>
> FWIW, the thing I miss sometimes is the equivalent of C's: 
>
>  while((x = next()) != nil) { 
>  something() 
>  } 
>
> In Go you need to do either: 
>
>  for x = next(); x != nil; x = next() { 
>  something() 
>  } 
>
> which duplicates the per-iteration expression, or: 
>
>  for { 
>   x = next() 
>   if x == nil { 
>   break 
>   } 
>   something() 
>  } 
>
> which is 4 lines longer and the invariant is inverted. 
>
> A two-part for statement could potentially do that: 
>
> for x = next; x != nil { 
> something() 
> } 
>
> but it's too subtly different from the normal form to be a good language 
> change. 
>

Funny you should mention that, because I proposed a very similar change 
back in March!
(https://golang.org/issue/24282)

It's almost certainly too invasive a change on its own, but if we happen to 
decide that a breaking change is the best way to address 
https://golang.org/issue/20733, I'm hopeful that it will at least be 
considered.


On 3 May 2018 at 08:18, Dan Kortschak  > wrote: 
> > Yeah, that's not `for {} else {}`. This is spelled 
> > 
> > ``` 
> > var done bool 
> > for condition() { 
> > done = true 
> > body() 
> > } 
> > if !done { 
> > outOfBody() 
> > } 
> > ``` 
> > 
> > On Wed, 2018-05-02 at 22:45 -0700, Sokolov Yura wrote: 
> >> 
> >> for { 
> >> Body() 
> >> if !Condition() { 
> >> break 
> >> } 
> >> } 
> >> 
> >> It is thats simple, guys. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: On Accepting Interfaces and Structs

['Bryan Mills' via golang-nuts]

I agree with Jake: a more complete example would be helpful. In my 
experience, this sort of issue is often a symptom of awkward package 
boundaries. If you can find a more natural package boundary, you may be 
able to sidestep the problem.

(There are certainly some cases where the lack of covariance makes 
interfaces difficult to construct, but those cases tend to be rare.)


On Monday, April 23, 2018 at 12:08:38 PM UTC-4, Jake Montgomery wrote:
>
> My gut feeling is that there is an elegant way to achieve what you want. 
> Unfortunately, I am having trouble following the code you are describing, 
> and more importantly,  what you are actually trying to accomplish. Perhaps 
> it is my failing.
>
> If you would post a more complete example, showing the relevant code in 
> all the related packages, I suspect you would get constructive suggestions.
>
> - Jake
>
>
> On Saturday, April 21, 2018 at 7:51:55 AM UTC-4, Kaveh Shahbazian wrote:
>>
>> Regarding "Accept interfaces, return concrete types", how can it be 
>> applied for structs that represent a payload/value?
>>
>> For example in package first, logger is defined as:
>>
>> type logger interface {
>> Debugf(template string, args ...interface{})
>> Errorf(template string, args ...interface{})
>> Infof(template string, args ...interface{})
>> }
>>
>> And package first only accepts a logger that implements the logger 
>> interface.
>>
>> Now lets assume there is a need for passing a struct too, like some 
>> config or state.
>>
>> This causes importing the original package that, that config or state 
>> struct resides in; while package first is happily accepting other things 
>> from that package using interfaces.
>>
>> For example in package second there is some tool that is represented 
>> using this interface in package first:
>>
>> type cloner interface {
>> Clone() (*second.State, error)
>> }
>>
>>
>> As it can be seen, now package first has to explicitly import package 
>> second, because of the type *second.State.
>>
>> Currently I break things by renaming the second package to something 
>> meaningless when importing like:
>>
>> type cloner interface {
>> Clone() (*p2nd.State, error)
>> }
>>
>> But this is not really a work around and package second leaks into the 
>> scope of package first anyway.
>>
>> Is there a way to actually achieve this?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Is there a memory leak on Rob Pike's demo codes?

['Bryan Mills' via golang-nuts]

To answer the original question about how to detect the leak: if you have a 
leak, the “goroutine” profile from runtime/pprof 
 will show an 
ever-increasing number of goroutines with the same stack trace.

Detecting such a leak in a unit or integration test, however, is fairly 
tricky: you can fairly easily detect that the goroutine count is 
increasing, but it can be tricky to figure out whether that is a leak or 
just a transient increase while the program spools up to a steady state.


On Thursday, April 5, 2018 at 4:53:21 PM UTC-4, T L wrote:
>
>
>
> On Thursday, April 5, 2018 at 3:32:53 AM UTC-4, Jan Mercl wrote:
>>
>> On Thu, Apr 5, 2018 at 9:10 AM T L  wrote:
>>
>> > yes. it is a resource leak.
>>
>> It's not necessarily a leak. It's a possible leak.
>>
>> And digging even deeper: it's implementation defined. Non-reachable 
>> channels can be collected and goroutines blocked on them killed without 
>> changing the semantics of the program.
>> -- 
>>
>> -j
>>
>
> There is an issue to do this, but it is denied. 
> https://github.com/golang/go/issues/19702
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: panic propagation can be blocked by blocking defer function

['Bryan Mills' via golang-nuts]

This is indeed a subtle pattern, but it's not clear to me how often it 
comes up in practice, and preventing deferred function calls from blocking 
could result in all kinds of other unwanted side effects.

I'm curious: how did you end up with the `wg.Add` call so far from the 
deferred `Wait`, and what triggers the corresponding `wg.Done`? (It seems 
unusual for the `Add` and `Done` calls not to have the same nesting 
structure as the potential panic and defer.)


On Saturday, March 31, 2018 at 8:28:18 PM UTC-4, Yongjian Xu wrote:
>
> This program will make panic not panicking but silently blocking forever.
>
> func panicfunc() {
> panic("oops")
> }
>
> func run() {
> defer func() {
>select {}
> }
> panicfunc()
> }
>
> func main() {
> // not trigger runtime deadlock.
> go func() {
>  ticker := time.NewTicker(10*time.Second) 
>  select {
>  case <-ticker.C:
>  }
> }()
> run()
> }
>
> The common understanding is that the only way to prevent panic from 
> propagating is to write "recover" function in the defer call chain, but due 
> to that panic allows defers to be executed normally, it seems that 
> accidentally blocking panicking propagation could become fairly trivial and 
> subtle.
>
> The above program is too obvious as an illustration, so it's likely not 
> happening very often or sneaking out of some careful eyes, but in real 
> situation, this might become much less obvious.  e.g. (stripped from our 
> production code)
>
> func (r *Resumer) FromCheckpoint(id string) {
>  r.mu.Lock()
>  defer r.mu.Unlock()
>  ... mutate other fields of r...
>  r.wg.Add(1)
> * r.resumeStarts[id] = time.Now()  *  
> }
>
> func (r *Resumer) WaitLoaded() {
>   r.wg.Wait()
> }
>
> func (r *Resumer) RecreateFromCheckpoint() {
>   for _, cp := r.checkpoint.Entries() {
> id := cp.GetId()
> r.FromCheckpoint(id)
> task := NewTask(id, cp)
> go task.Run()
>   }
> }
>
> func (r *Runner) Start() {
>defer resumer.WaitLoaded()
>
>... other logic ...
>if err := resumer.RecreateFromCheckpoint(); err != nil {
>   os.Exit()
>}
> 
>... create RPC server...
>... register HTTP handler...
>go server.Serve()
> }
>
> We added a new field in Resumer "resumeStarts" but forgot to put code to 
> correctly initiate it, so when the code runs, it should panic, but 
> unfortunately, it doesn't because we have a r.wg.Add(1) statement* 
> *before** the map assignment and the wg.Wait is under a defer path, which 
> will block panic propagation.
>
> The most easiest fix is to just write
>
> func (r *Resumer) FromCheckpoint(id string) {
>  r.mu.Lock()
>  defer r.mu.Unlock()
>  ... mutate other fields of r...
> * r.resumeStarts[id] = time.Now()*
>  r.wg.Add(1)
> }
>
> Completely arbitrary order without altering the program semantics and 
> correctness but yet, write this way would not block the panic.
>
> Because the symptom is a completely block and there is absolutely no 
> recover call through out the stack, it was *SO NOT OBVIOUS* before we 
> found the issue.
>
> I am wonder if there is anything we can do to improve runtime support on 
> panic propagation to make sure that other than the builtin "recover", 
> nothing could prevent a panic unwinding the stack.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Experience report on coming to Go from a C perspective

['Bryan Mills' via golang-nuts]

You might be interested in https://golang.org/issue/12854 for maps, slices, 
and structs.

(It wouldn't help with channels, because there is currently no such thing 
as a channel literal.)

On Thursday, February 22, 2018 at 5:32:40 PM UTC-5, Devon H. O'Dell wrote:
>
> 2018-02-22 14:19 GMT-08:00 Caleb Spare : 
> > I occasionally run into this, but in my experience it's exclusively with 
> > maps: 
>
> I think all of the cases I ran into this were for maps and channels. 
>
> > - Initializing empty slices is ~never necessary; (in my opinion the 
> > "members: []int{}" from the blog post is a code smell). 
>
> Probably, but as mentioned in the post, this example was to illustrate 
> the point. It was also to keep consistency with the C example. This 
> would have been convoluted to do for maps or channels. 
>
> > - Channels need initialization but that's where the important buffered 
> vs. 
> > unbuffered choice is made, so I'm happy that's explicit. (I suppose you 
> can 
> > say the same about initializing maps with a size, but that's fairly rare 
> in 
> > my experience.) 
>
> That's fair, but why require expressing the type in the call to make? 
> You already know the type of the channel; you should only need to 
> specify the buffer length (if any). Type inference is so pervasive 
> through the rest of the language; the compiler should absolutely be 
> able to help here. 
>
> For example: 
>
> return {c: make(chan T)} 
> return {c: make(chan T, 100)} 
> return {m: make()} 
> return {m: make(100)} 
>
> and 
>
> return {c: make()} 
> return {c: make(100)} 
> return {m: make()} 
> return {m: make(100)} 
>
> I think the latter set of examples is much nicer. Yes, you don't see 
> _what_ make is allocating, but you knew that if you looked at the 
> specification for the type (which you probably already did). Other 
> idiomatic expressions already hide this information. For example, c := 
> foo.c also doesn't tell you what the type of the assignment is. And 
> bar := foo is even more indirect. 
>
> --dho 
>
> > - If the struct has a nested pointer to another struct type, then having 
> it 
> > be nil is definitely what you want since the nested type may or may not 
> have 
> > a valid zero value. 
> > 
> > On Thu, Feb 22, 2018 at 12:02 PM, Devon H. O'Dell  > 
> > wrote: 
> >> 
> >> Hi all, 
> >> 
> >> It's been some time since I really contributed much of anything to the 
> >> project (sorry!), but after 8 years, I'm finally writing Go outside of 
> >> the project itself (and outside of porting efforts). I was lamenting 
> >> to some coworkers about the lack of a comparable feature to C's 
> >> "malloc idiom" and they suggested I write an experience report on it. 
> >> I wrote the bulk of the article a month ago, but finally put in some 
> >> finishing touches and published. 
> >> 
> >> For whatever it's worth (probably not much): 
> >> https://9vx.org/post/a-malloc-idiom-in-go/ 
> >> 
> >> --dho 
> >> 
> >> -- 
> >> You received this message because you are subscribed to the Google 
> Groups 
> >> "golang-nuts" group. 
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an 
> >> email to golang-nuts...@googlegroups.com . 
> >> For more options, visit https://groups.google.com/d/optout. 
> > 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] context.Context and worker pools

['Bryan Mills' via golang-nuts]

Pool the resources, not the workers.

For example, if your resource is “memory footprint of in-flight requests”, 
use a semaphore  instead. 
(There is a “worker pool” example in the package docs.)

If your resource is a fixed-size set of connections, you can store the 
connections in a buffered channel and select on acquiring it or ctx.Done().


On Friday, February 16, 2018 at 6:30:35 PM UTC-5, Bruno Albuquerque wrote:
>
> I did something like this here:
>
> https://github.com/brunoga/workerpool
>
> I do not store the context anywhere, I pass it to my Start() method and it 
> passes it down to any place I need it.
>
>
> On Fri, Feb 16, 2018 at 1:05 PM andrey mirtchovski  > wrote:
>
>> While trying to retrofit context.Context within a
>> worker-pool-patterned package, where work is sent down a channel to be
>> picked up by number of worker goroutines.
>>
>> I'm running against the mantra of "Do not store Contexts inside a struct 
>> type".
>>
>> For example, I want to put a timeout on the amount of time spent
>> waiting to write on a channel, plus the time it takes for my write to
>> be consumed on the other end (the time the requests spends enqueued on
>> the channel buffer). How would I go about doing that without embedding
>> a context.Context inside my write?
>>
>> A typical example, here is how i would do it without the channel send:
>>
>> func p(ctx context.Context, req Request) {
>>  ctx, _ = context.WithTimeout(ctx, Timeout)
>>  select {
>>  case <-ctx.Done:
>>   // timed out
>>  default:
>>  worker(ctx, req) // will use ctx to determine if timeout expired
>>  }
>> }
>>
>> With a channel I may do something like:
>>
>> func c(req Request, in chan Request) {
>>   in <- req
>>   <-out
>> }
>>
>> func worker(in chan Request) {
>>  for _, v := range in {
>>  // i want to cancel here if 'v' has lived past its timeout
>>  // or do work if not
>>  }
>> }
>>
>> And I want to have a meaningful way to measure enqueue time before req
>> is consumed by a worker in the second example.
>>
>> Hopefully that makes sense.
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to pass Go []byte (arrays/slices) as C *char to C functions

['Bryan Mills' via golang-nuts]

It may be considerably more efficient to pass the Go pointers directly.
To do so, you may need to convert through unsafe.Pointer.

Note that many C APIs that take a char* expect it to be null-terminated, so 
if you take that approach you may need to append an extra 0 to the slice.

https://golang.org/cl/56530 has some examples.


On Tuesday, January 23, 2018 at 11:31:44 AM UTC-5, Jan Mercl wrote:
>
> On Tue, Jan 23, 2018 at 5:02 PM Christian LeMoussel  > wrote:
>
> > Is it correct /best way ? 
>
> Hopefully/only. But as noted previously, not tested.
>   
> > There is no memory leaks ?
>
> Something must eventually C.free() the memory allocated by C.CString().
>
>
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is it reasonable?

['Bryan Mills' via golang-nuts]

On Sunday, January 7, 2018 at 6:09:45 AM UTC-5, Jan Mercl wrote:
>
> On Sun, Jan 7, 2018 at 11:57 AM T L  
> wrote:
>
> Yes. The choice is to raise a runtime error or not. When not, the value 
> does not really matter because every value of an integer variable 
> represents an integer number and there is no combination of the bits left 
> unused for NaN, Inf, etc. FTR, the value is what the CPU produces. The CPU 
> sets flags on the conversion but those are not available directly to the 
> program. The comma-ok idiom could be useful in such cases, though.
>

There is not currently a comma-ok form for integer type conversions.
The proposal in https://golang.org/issue/19624 would add one.

Non-representable floating point values have to be checked before the 
> conversion where appropriate.
>

The spec  says:
“In all non-constant conversions involving floating-point or complex 
values, if the result type cannot represent the value the conversion 
succeeds but the result value is implementation-dependent.”

So you can check the value after the conversion, too, and that check is a 
bit simpler than trying to cover all of the cases beforehand:

y := int64(x)
if float64(y) != x {
…
}

>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Channels vs Callbacks in API

['Bryan Mills' via golang-nuts]

In some cases, a synchronous callback can be a fine choice: consider 
filepath.Walk .

A synchronous callback does not require any extra goroutines, and if the 
caller needs to adapt it to work with a channel (or perform longer-duration 
processing in parallel) that's easy enough to do at the call site: then the 
information about blocking/buffering/closing/draining behavior is localized 
at the call side instead of split across multiple packages.


As Axel notes, a channel-based API adds a lot of complexity due to 
goroutines: you have to be careful not to leak them, particularly if the 
caller stops reading early (e.g. due to encountering an error).

However, goroutines are not the only issue: in particular, you should also 
consider the interaction with error reporting. What happens if the function 
encounters an error? How will you inform the caller, will they remember to 
check for it, and will forgotten error-checks be obvious to future readers 
of the code? With an iterator or synchronous-callback API, the function 
call can return the error directly. With channels, you have to either 
return a channel of value-or-error or return a separate `<-chan error`, and 
either alternative makes the channel-draining invariants even more complex 
(and goroutine leaks more likely).

For example, with a separate error channel, if one channel is closed the 
caller needs to remember whether to check/drain the other one. With a 
single channel of value-and-errors, the caller needs to know whether they 
should keep reading the channel after the first error.

On Saturday, December 30, 2017 at 11:51:39 AM UTC-5, thepud...@gmail.com 
wrote:
>
> On a semi-related note, I think there is a fair amount of wisdom in the 
> "Synchronous Functions" section of the Code Review Comments golang wiki 
> page (including it is not written as an absolute and starts with the word 
> "Prefer").
>
> https://github.com/golang/go/wiki/CodeReviewComments#synchronous-functions
>
>  
>
> --
> Synchronous Functions
> --
> Prefer synchronous functions - functions which return their results 
> directly or finish any callbacks or channel ops before returning - over 
> asynchronous ones.
>
> Synchronous functions keep goroutines localized within a call, making it 
> easier to reason about their lifetimes and avoid leaks and data races. 
> They're also easier to test: the caller can pass an input and check the 
> output without the need for polling or synchronization.
>
> If callers need more concurrency, they can add it easily by calling the 
> function from a separate goroutine. But it is quite difficult - sometimes 
> impossible - to remove unnecessary concurrency at the caller side.
> --
>
>
> --thepudds
>
> On Friday, December 29, 2017 at 6:41:43 PM UTC-5, dc0d wrote:
>>
>> 1 - I should emphasize that the API in question is concurrent by nature 
>> (inbounds at any time which might need replies).
>>
>> 2 - All points made are good valid points in general. Thanks!
>>
>> 3 - With 1 in mind, still IMHO, channels should be preferred. Draining on 
>> either side, needs goroutines. But consumer must have better control - not 
>> just one goroutine per inbound. But yes, it's a concern.
>>
>> (I'm on mobile now, but I'll give it more thought).
>>
>> On Sat, Dec 30, 2017, 02:00 Axel Wagner  
>> wrote:
>>
>>> Some counterpoints:
>>>
>>> * In the question of "Channels vs. Callbacks", the best choice is 
>>> "neither". Write a blocking API with a fixed number of items and a 
>>> Scanner-like Iterator for an unknown number of items.
>>> * It requires spawning a Goroutine even if none is necessary
>>> * It adds syntactic overhead to the caller even if none is necessary
>>> * If the caller in any way wants to customize the channel (e.g. add 
>>> buffering) they have to spawn yet another Goroutine
>>> * It requires specifying how the API behaves if the channel isn't 
>>> read/quickly enough/whether it needs to be drained or not/when it will be 
>>> drained - while an io.Closer already is the common idiom for resources to 
>>> be closed
>>>

- Internals of the API might be affected by unknown behavior of 
callbacks 

 Like what? A panic should just crash the program, no affected behavior. 
>>> The package providing the API shouldn't have to worry about it. Anything 
>>> else is isomorphic to unknown behavior of the channel consumer.
>>> That being said, again, the correct choice is neither, which also 
>>> doesn't suffer any of these problems. Encapsulate the concurrency.
>>>

- lets the consumer decides how many goroutines is needed to handle 
it

 Just like a Scanner-like iterator.
>>>
>>

-- 
You received this message because 

[go-nuts] Re: Sending C array over gob ; decoding shows wrong data

['Bryan Mills' via golang-nuts]

Per https://golang.org/pkg/encoding/gob/, “Structs encode and decode only 
exported fields.”
The fields of your C struct are unexported in Go, since they start with 
lower-case letters.

If the gob package is encoding or decoding the non-array fields of the 
struct, *that* is the bug.


On Monday, December 18, 2017 at 8:04:32 AM UTC-5, Tamás Gulácsi wrote:
>
> Use a Go struct for GOB encoding/decoding.
>
> 2017. december 18., hétfő 13:51:26 UTC+1 időpontban Kanika Kakkar a 
> következőt írta:
>>
>> I am sending a C structure which looks like this
>>
>> typdef struct 
>> {
>>  unsigned char field1;
>>  unsigned char field2;
>>  unsigned char array[10];
>> }complete_data_t;
>>
>> I am encoding this data using GOB and sending as UDP broadcast in a very 
>> small network. On the receiver side, all fields are correctly getting 
>> parsed but all array elements are coming as zero. I am populating these 
>> array elements with a non-zero value at the transmitter side.
>>
>> var data C.complete_data_t
>>
>> Transmitter side:
>> var buffer bytes.Buffer
>> err := gob.NewEncoder().Encode()
>>   _,err = conn.Write(buffer.Bytes())
>> buffer.Reset()
>>
>> Receiver side:
>> rx_buf := make([]byte, 1024)
>> length,addr,err := conn.ReadFrom(rx_buf)
>> buffer := bytes.NewBuffer(rx_buf[:length])
>> err = gob.NewDecoder(buffer).Decode()
>>
>>
>>   
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Mapping C pointer to BYTE to golang []byte

['Bryan Mills' via golang-nuts]

In this case, the vet tool is correct if you're making the syscall with 
Go-allocated memory.
The Go runtime is allowed to move values around: the address of a Go 
variable is only pinned for the duration of the syscall itself.

If you've got C-allocated memory (or statically-allocated 
memory), https://golang.org/issue/13656#issuecomment-303216308 has a 
solution that avoids copying and is more robust to large sizes.


On Thursday, December 14, 2017 at 5:27:57 AM UTC-5, snmed wrote:
>
> Okay I found a way, this seems to work:
>
> ca := (*[100]byte) (unsafe.Pointer(certctx.pbCertEncoded))[:certctx.
> cbCertEncoded]
>
> or
>
> mm := make([]byte, certctx.cbCertEncoded)
> for i := uint32(0) ; i < certctx.cbCertEncoded; i++ {
> mm[i] = *((*byte)(unsafe.Pointer(certctx.pbCertEncoded + uintptr(i
> ) * unsafe.Sizeof(new(byte)
> }
>
>
>
> Anyway, the vet tool is complaining:
>
> main.go:106: possible misuse of unsafe.Pointer
> main.go:109: possible misuse of unsafe.Pointer
>
> This is the code fragment:
>
> 104certctx := new(CERT_CONTEXT)
> 105
> 106certctx = (*CERT_CONTEXT) (unsafe.Pointer(pccert_context))
> 107
> 108
> 109ca := (*[100]byte) (unsafe.Pointer(certctx.pbCertEncoded))[:
> certctx.cbCertEncoded]
>
> Is there another way to use syscall return values uintptr without vet 
> warnings? And which solution should I prefer? I think the later one should 
> be more safe, isn't it?
>
> Cheers
>
>
> Am Donnerstag, 14. Dezember 2017 09:29:38 UTC+1 schrieb snmed:
>>
>> Hi Miki 
>>
>> I'm using syscall package and no C import, but maybe it is possible to 
>> use this function as well?
>>
>> Am Donnerstag, 14. Dezember 2017 09:18:26 UTC+1 schrieb Miki Tebeka:
>>>
>>> Do you mean
>>>
>>> func C.GoBytes(unsafe.Pointer, C.int) []byte
>>>
>>>  ?
>>>
>>> On Thursday, December 14, 2017 at 9:05:32 AM UTC+2, snmed wrote:

 Hi all

 I'm trying to map a C structure to an equivalent go struct, but I 
 bumped into a problem with a pointer to byte that is actually an array of 
 bytes.

 Here is the C struct:

 typedef struct _CERT_CONTEXT {
   DWORD  dwCertEncodingType;
   BYTE   *pbCertEncoded;
   DWORD  cbCertEncoded;
   PCERT_INFO pCertInfo;
   HCERTSTORE hCertStore;
 } CERT_CONTEXT, *PCERT_CONTEXT;


 and this is my go struct:

 type CERT_CONTEXT struct {
 dwCertEncodingType uint32
 pbCertEncoded  uintptr
 cbCertEncoded  uint32
 pCertInfo  uintptr
 hCertStore uintptr
 }

 for my case I need only the first 3 fields and I do not have any 
 problem to get 1 and 3, but I can't remember how to translate the second 
 field to a slice of bytes.
 This is how I map the struct from an uintptr and print it to the 
 console:

 certctx = (*CERT_CONTEXT) (unsafe.Pointer(pccert_context))
 fmt.Printf("%v\n", certctx)
 
 >&{1 807520 674 833008 789360}

 Any advise is warmly welcome.

 Cheers,
 Sandro

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Should an empty append be an error ?

['Bryan Mills' via golang-nuts]

The spec defines `append` as a variadic function, with a type of the form 

append(s S, x ...T) S  // T is the element type of S

Variadic functions, in general, accept zero or more parameters. (Consider 
fmt.Println().)

In order to reject empty appends, append would either have to have its own 
special type structure (not just a variadic function), or it would have to 
be of the form

append(s S, x T, xs ...T) S

The former would increase the complexity of the language spec for little 
benefit. The latter would break call sites that look like

foo = append(foo, bar...)

 
As Axel and Dave suggest, this would probably be reasonable for a linter 
(or cmd/vet) to warn about.


On Sunday, December 3, 2017 at 7:09:39 PM UTC-5, nz wrote:
>
> My point is that it should give an error or refuse to compile.
>
> On Sunday, 3 December 2017 15:10:42 UTC-8, Dave Cheney wrote:
>>
>> What is the error? This construct seems harmless.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] chan chan Job per worker vs. single chan Job

['Bryan Mills' via golang-nuts]

Why have long-lived worker goroutines at all?

If you're just trying to limit the number of jobs in flight, a semaphore is 
usually simpler.
(A `chan struct{}` or `chan bool` works fine as a simple semaphore; for 
more complex use-cases, there are several semaphore packages on godoc.org. 
Personally, I'm partial to https://godoc.org/golang.org/x/sync/semaphore.)

If the workers need some resource that is only available in fixed 
quantities, it's still usually simpler to store and distribute the 
resources (e.g. in a buffered channel) rather than the jobs.

As rog notes, a more complete example might be helpful.


On Friday, November 24, 2017 at 12:57:04 PM UTC-5, Michael Jones wrote:
>
> having workers pull from a single queue is an excellent approach. (unless 
> you pull more than 3 million items per second from that queue :-)
>
> On Fri, Nov 24, 2017 at 2:11 AM, roger peppe  > wrote:
>
>> On 24 November 2017 at 05:46,   
>> wrote:
>> > Hi!
>> >
>> > This should be covered somewhere already but I can't find it. So here's 
>> my
>> > question:
>> >
>> > Assume N workers running in a goroutine each and a number of Job tasks
>> > coming in.
>> >
>> > The consensus appears to be to have one dispatcher goroutine collecting 
>> all
>> > jobs, then pulling a ready worker from a single queue of worker 
>> channels.
>>
>> What do you base that consensus on? Having all workers pull from
>> a single shared channel seems like a fine way to do things.
>>
>> It would help if you could provide some more complete code - the
>> snippets you provided don't really make sense on their own to me.
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Michael T. Jones
> michae...@gmail.com 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How to convert from []byte to []uint32?

['Bryan Mills' via golang-nuts]

Yes, it looked like a typo to me.

Truncating the string to an even number of characters gives the same 
results as the original snippet:
https://play.golang.org/p/avf6xqnpEn

It's also not difficult to pad the input, if that's the desired behavior:
https://play.golang.org/p/Q6C9SBhQB5
(Padding without copying the whole string would be slightly more complex, 
but as Howard noted, Go style is generally to prefer simpler code.)


On Monday, November 13, 2017 at 5:31:22 PM UTC-8, peterGo wrote:
>
> Bryan,
>
> Use the test case from the question: buffer := "83f982d600c1caca7a6".
>
> https://play.golang.org/p/1gN7Y4ajOH
>
> Peter
>
> On Monday, November 13, 2017 at 7:42:16 PM UTC-5, Bryan Mills wrote:
>>
>> In this case, the "code golf" solution seems clearer: 
>> https://play.golang.org/p/Jxkf2Vheml
>>
>> On Monday, November 13, 2017 at 3:57:57 PM UTC-8, peterGo wrote:
>>>
>>> Christian,
>>>
>>> Your specialized convertCharToInt32 function, which returns []uint32, is 
>>> slow in comparison to a more general HexToUint function.
>>>
>>> BenchmarkHexToUint32-8   200088.9 ns/op   16 B/op2 allocs/op
>>> BenchmarkCharToInt32-8300   438 ns/op 96 B/op   22 allocs/op
>>>
>>> Playground: https://play.golang.org/p/OeUDEV9Xpb
>>>
>>> Peter
>>>
>>> On Monday, November 13, 2017 at 1:51:21 AM UTC-5, Christian LeMoussel 
>>> wrote:

 I have a data stream of bytes and I'd like to get array of int32 (from 
 four bytes).

 func convertCharToInt32(buffer string) []uint32 {
 const SIZEOF_INT32 = 4

 var hh = make([]byte, 2)
 var cbuffer = make([]byte, len(buffer)/2)
 var hbuffer = make([]uint32, len(cbuffer)/SIZEOF_INT32)

 for i := 0; i < 28; i++ {
 hh[0] = buffer[i*2]
 hh[1] = buffer[i*2+1]
 if s, err := strconv.ParseUint(string(hh[:]), 16, 64); err == 
 nil {
 cbuffer[i] = byte(s)
 }
 }

 for i := range hbuffer {
 hbuffer[i] = uint32(Endian.Uint32(cbuffer[i*SIZEOF_INT32 : (i+1
 )*SIZEOF_INT32]))
 }

 return hbuffer
 }

 buffer := "83f982d600c1caca7a6"
 hbuffer := convertCharToInt32(buffer)



 The code above seems to work, but perhaps there is a built-in function 
 in Go that I've missed or there is a super cool hack that does that in one 
 instruction?




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: How to convert from []byte to []uint32?

['Bryan Mills' via golang-nuts]

In this case, the "code golf" solution seems 
clearer: https://play.golang.org/p/Jxkf2Vheml

On Monday, November 13, 2017 at 3:57:57 PM UTC-8, peterGo wrote:
>
> Christian,
>
> Your specialized convertCharToInt32 function, which returns []uint32, is 
> slow in comparison to a more general HexToUint function.
>
> BenchmarkHexToUint32-8   200088.9 ns/op   16 B/op2 allocs/op
> BenchmarkCharToInt32-8300   438 ns/op 96 B/op   22 allocs/op
>
> Playground: https://play.golang.org/p/OeUDEV9Xpb
>
> Peter
>
> On Monday, November 13, 2017 at 1:51:21 AM UTC-5, Christian LeMoussel 
> wrote:
>>
>> I have a data stream of bytes and I'd like to get array of int32 (from 
>> four bytes).
>>
>> func convertCharToInt32(buffer string) []uint32 {
>> const SIZEOF_INT32 = 4
>>
>> var hh = make([]byte, 2)
>> var cbuffer = make([]byte, len(buffer)/2)
>> var hbuffer = make([]uint32, len(cbuffer)/SIZEOF_INT32)
>>
>> for i := 0; i < 28; i++ {
>> hh[0] = buffer[i*2]
>> hh[1] = buffer[i*2+1]
>> if s, err := strconv.ParseUint(string(hh[:]), 16, 64); err == nil 
>> {
>> cbuffer[i] = byte(s)
>> }
>> }
>>
>> for i := range hbuffer {
>> hbuffer[i] = uint32(Endian.Uint32(cbuffer[i*SIZEOF_INT32 : (i+1)*
>> SIZEOF_INT32]))
>> }
>>
>> return hbuffer
>> }
>>
>> buffer := "83f982d600c1caca7a6"
>> hbuffer := convertCharToInt32(buffer)
>>
>>
>>
>> The code above seems to work, but perhaps there is a built-in function in 
>> Go that I've missed or there is a super cool hack that does that in one 
>> instruction?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: NaN as key in map

['Bryan Mills' via golang-nuts]

On Friday, November 10, 2017 at 5:24:25 PM UTC-5, Stefan Nilsson wrote:
>
> NaN is not equal to NaN. This means that if you use NaN as a key in a 
> map[float64]string you will not be able to retrieve it again: 
> https://play.golang.org/p/A7KGgoM3o6
> I guess this is the way it should be, but it's still a little bit 
> disconcerting. What's your thoughts on this?
>

See https://golang.org/issue/20660. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] is this code thread safety?

['Bryan Mills' via golang-nuts]

In general you can use the race detector to help answer this sort of 
question, although it does not detect all races.

Unfortunately, it doesn't detect this one. (I've 
filed https://golang.org/issue/22569, because I think it should.)

On Friday, November 3, 2017 at 4:08:19 AM UTC-4, sheepbao wrote:
>
> Thanks, I got it.
>
> On Friday, November 3, 2017 at 11:26:31 AM UTC+8, Jesse McNelis wrote:
>>
>> On Thu, Nov 2, 2017 at 4:54 PM, sheepbao  wrote: 
>> > 
>> > the close function is thread safety? how about call `closed` at the 
>> same 
>> > time. 
>>
>> It's not safe. Multiple goroutines can enter the 'default'  case in 
>> that select and close() the channel multiple times. 
>>
>> "Sending to or closing a closed channel causes a run-time panic." 
>> https://golang.org/ref/spec#Close 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to know if interface{} data is nil w/o reflecting?

['Bryan Mills' via golang-nuts]

On Wednesday, November 1, 2017 at 10:32:21 AM UTC-4, Ian Lance Taylor wrote:
>
> On Wed, Nov 1, 2017 at 4:18 AM,   wrote: 
> > 
> > Ayan, imagine I am part of a development team. In our program I have a 
> > pointer r: 
> > 
> > r *myType 
> > 
> > My variable r can be nil, because that is a valid situation. It is used 
> in 
> > dozens of places, like this: 
> > 
> > if r != nil { 
> > r.doSomething() 
> > } 
> > 
> > That is a very usual idiom, no only in Go, but in many languages. Every 
> > programmer is acquainted to that. 
> > 
> > Then, years later, other programmer in my team decides to create an 
> > interface to better capture a new understanding of the problem at hand, 
> > changing the type of r to: 
> > 
> > r myInterface 
> > 
> > Subtly, the meaning of 
> > 
> > if r != nil { 
> > r.doSomething() 
> > } 
> > 
> > changes. Under the right circumstances our software starts to behave 
> > strangely. What? 
> > 
> > This problem is dangerous because it is so subtle. We will read our old 
> code 
> > time and again to no avail, because everything seems fine and no one has 
> > changed that "if r != nil r.doSomething" in ages. 
> > 
> > As Dave Cheney said in 
> > https://github.com/golang/go/issues/21538#issuecomment-323561094 that 
> could 
> > be solved: 
> > 
> > "by making an nil interface and an interface which contains a type who's 
> > value is nil also equal to nil." 
>
> A different way to state your argument is that nil is overloaded in 
> Go.  Your argument is essentially identical to saying suppose you have 
> an integer r: 
>
> type MyIntegerType int 
> var r MyIntegerType 
>
> and a bunch of code that tests whether r is 0 or not 
>
> if r != 0 { 
> r.doSomething() 
> } 
>
> Then you change r to an interface type, and now the meaning of r != 0 
> has subtly changed.  The code still compiles, but now it means 
> something different.  r != 0 will be true even if the value of r 
> actually is 0, because in `r != 0` the `0` will be given type `int`, 
> but `r` is type `MyIntegerType`, so the values are not equal. 
>
> That is definitely potentially confusing, but I would like to believe 
> that you would not argue that we should treat r != 0 specially for 
> integer types.  Or, at least, we shouldn't treat it any differently 
> than r != 1.  And, obviously, `r == nil` should not be true if r 
> happens to be `0`. 
>
> I think the reason it feels different for pointer types is that we, 
> probably incorrectly, chose to use the same identifier, `nil`, to 
> represent the zero value both for pointers and for interfaces. 
> Suppose we had given the zero value of interface types the name 
> `nilinterface`.  Then the situation for pointers would be the similar 
> to that for integers.  Writing `r != nilinterface` would be equivalent 
> to `r != nil` today: it would test whether the interface itself is 
> nil.  `r != nil` would be a compilation error, because, unlike `0`, 
> there is no default type for `nil`. 
>
> If we do change something in this area for Go 2, which we probably 
> won't, my preference would definitely be to introduce `nilinterface` 
> rather than to further confuse the meaning of `r != nil`. 
>

C++ had a similar change with the `nullptr 
` literal in C++11: 
`NULL` was overloaded between pointers and integers, whereas `nullptr` can 
only be (implicitly converted to) a pointer.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Make error handling less verbose with Zig %% operator and %return expression?

['Bryan Mills' via golang-nuts]

There are lots of similar proposals in the issue tracker.
Of those, https://golang.org/issue/21161 seems to have the most traffic.

On Tuesday, October 17, 2017 at 5:09:33 PM UTC-4, Nicolas Grilly wrote:
>
> I just read about Zig, a new programming language designed to replace C.
>
> The error handling philosophy is very similar to Go: control flow in 
> general and error handling in particular is explicit. Zig has even a defer 
> statement inspired by Go ;-)
>
> I noticed that Zig provides a %% operator and %return expression meant to 
> make error handling less verbose:
>
> http://ziglang.org/documentation/master/#errors
>
> I'd be curious to know if something similar would be possible in a future 
> version of Go?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Compile-time parameters in Zig: a possible solution to generic functions and data structures in Go?

['Bryan Mills' via golang-nuts]

On Tuesday, October 17, 2017 at 4:52:55 PM UTC-4, Nicolas Grilly wrote:
>
> Hello,
>
> I just read about the concept of compile-time parameters in Zig (a new 
> programming language designed to replace C):
>
>
> http://andrewkelley.me/post/zig-programming-language-blurs-line-compile-time-run-time.html
>
> Could it be a way to provide generic functions and data structures in Go?
>
> I note we already have some kind of "compile-time feature" with constants.
>
> Nicolas Grilly
>


See 
https://github.com/golang/proposal/blob/master/design/15292/2016-09-compile-time-functions.md
 
for a similar proposal for Go.

(Note that in my proposal, “compile-time” is a property of the function 
rather than the parameter. From what I can tell the implications of that 
difference are minor.)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Sync pool old data

['Bryan Mills' via golang-nuts]

On Wednesday, October 11, 2017 at 4:03:30 PM UTC-4, Ian Lance Taylor wrote:
>
> if the number of parallel operations does not vary 
> significantly, a freelist is a simpler and more effective technique. 
>

A freelist might be simpler, but it isn't necessarily more effective. 
sync.Pool combines two (non-orthogonal) properties: thread-local allocation 
and weak references.

If the number of operations does not vary, the weak references don't matter 
much, but the thread-local allocation might. If many operations occur in 
parallel on a system with many CPU cores, a freelist can suffer from 
significant cache contention, whereas a sync.Pool generally will not. (See 
also https://golang.org/issue/18802.)

That said, Go's allocator is fairly thread-local to begin with, so if 
you're explicitly zeroing out the objects before reusing them anyway, 
sync.Pool should provide very little benefit over simply allocating a new 
object for each operation. (That is especially true when the objects are 
small, and thus more likely to be satisfiable out of memory the thread has 
already obtained.)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] when to use Sync pool

['Bryan Mills' via golang-nuts]

On Friday, October 6, 2017 at 5:32:06 PM UTC-4, Ian Lance Taylor wrote:
>
> On Fri, Oct 6, 2017 at 1:23 PM, XXX ZZZ  
> wrote: 
> > 
> > So we are making a platform where we have to use a TON of short lived 
> > structs, in order to optimize this we intend to use sync pool, however 
> on 
> > our benchmark tests I have found that it only seems to improve 
> performance 
> > when the struct is big enough, for smaller structs ie: (4-5 string 
> values) 
> > it seems to be considerably slower, yet I see many packages (such as 
> > fasthttp) using sync pool for basically everything. Is there anything I 
> am 
> > missing? Any recommendations of when to use sync pool? 
>
> Be careful in your benchmarks.  Make sure that the values are really 
> being allocated, and not simply being stored on the stack by the 
> compiler.  Check how often the GC is running; allocation that does not 
> trigger a GC is always going to be faster. 
>
> Use sync.Pool when you have code that uses an unpredictable number of 
> objects, where the allocation cost of the objects is noticeable 
> compared to the cost of the rest of the code (either because the 
> objects are large or because the rest of the code is fast), where you 
> want to scale the allocation require based on the use of the package. 
> Don't use sync.Pool if the number of objects is predictable: use a 
> free list.  Don't use it for basically everything; it won't actually 
> save you any time.  Use it for areas where the memory needs change 
> unpredictably during the course of the program.


One of the interesting things I've noticed with sync.Pool is that the cost 
of boxing an unboxing the `interface{}` argument can sometimes swamp out 
the savings from pooling the objects: the interaction with escape analysis 
is particularly subtle.

You may be tempted to type-assert the return value from `Get` immediately 
to keep the code clean, like this:

```
p, _ := pool.Get().(*someType)
if p == nil {
p = new(someType)
}
…
pool.Put(p)
```

If your benchmarks turn out anything like mine did, you'll instead want 
something like:

```
var (
p *someType
i = pool.Get()
)
if i == nil {
p = new(someType)
i = p
} else {
p = i.(*someType)
}
…
pool.Put(i)
```

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.