Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
Here is a golang version of Daniel Bernstein's "eratspeed", which is a straight translation of his C code to golang without any changes other than as necessary to make it work: https://play.golang.org/p/Sd6qlMQcHF. It won't run on the golang playground because it takes too long unless one changes the number of loops it runs, but at least one gets a nicely formatted listing. I haven't played with optimizing this other than to turn of array bounds checks in compiling it (to make it more comparable to C), but it seems that "as-is" it runs at about the same speed as John's "primespeed" code from the beginning of this thread when that code has multi-threading disabled (force numprocs = 1 in the sieve.go file) when both have array bounds checking turned off, and both for these conditions run slightly faster than Bernstein's C "primespeed" in 32-bit mode just as compiled with GCC using his "make". Bernstein's C "eratspeed" under those some conditions runs about 17% slower than his "primespeed", which makes it about 20% slower than this golang version. Those are all under the conditions of being compiled and run on my machine, which isn't particularly fast of efficient. Neither have I yet looked at the machine code generated by this version of eratspeed, but it is likely helped by there being more constants so there is less of a demand for registers. This isn't the ultimate in Sieve of Eratosthenes algorithms yet as it could be maximally factorized (use a 2/3/5/7 wheel instead of the 2/3/5 one used and use a pre-cull pattern which has the 11/13/17/19 prime composites eliminated to initialize the culling array) to save about another 25% to 30%, multi-threading could be applied for another speed up by a factor of four on 4 cores plus HT CPU's; it also needs some work to make the range extendable since, as it is currently written, it is hard coded to sieve up to this specific range. Anyway, compare apples with apples, both algorithms true to those written by Bernstein. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Currying in Go
I would also like to hear from somebody about a real world use-case. I don't pretend to be proficient in this realm of functional programming, but I would be very surprised if this is valuable in a language like Go that can and does hold state. That said, this is very cool and I would love to be proven wrong. On Friday, June 17, 2016 at 7:34:04 AM UTC-7, Evan Digby wrote: > > Currying is translating the evaluation of a function with multiple > arguments into evaluating a sequence of functions with one argument. Not > sure how this doesn't qualify, even if a closure was used to accomplish > this. > > As for the value, at the very least there is the same value as using > closures in general. The rest (why converting to single argument functions > for pure Currying was necessary) would have to be use-case specific. The > example given doesn't speak to why it's valuable. > > I would be curious to understand the value by exploring more real-world > use-cases myself! > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: go1.7beta2 prints "too many values in struct initializer" when I ran "go test" for gopkg.in/vmihailenco/msgpack.v2
go test -work should do it. On Saturday, 18 June 2016 12:02:03 UTC+10, Hiroaki Nakamura wrote: > > Hi, > > Here are commands I used to set up gopkg.in/vmihailenco/msgpack.v2 > > go get -u gopkg.in/vmihailenco/msgpack.v2 > cd $GOPATH/src/gopkg.in/vmihailenco/msgpack.v2 > go get -u github.com/ugorji/go-msgpack > go get -u github.com/ugorji/go/codec > go get -u gopkg.in/check.v1 > > > And I tried to run tests and got the following errors. > > $ go test -v > # testmain > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:112: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:114: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:116: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:118: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:120: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:122: > too many values in struct initializer > /tmp/go-build808637166/ > gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go:124: > too many values in struct initializer > FAILgopkg.in/vmihailenco/msgpack.v2 [build failed] > > > I tried to look into the > /tmp/go-build808637166/gopkg.in/vmihailenco/msgpack.v2/_test/_testmain.go > but the directory /tmp/go-build808637166 didn't exist. > > $ ls /tmp/go-build808637166 > ls: cannot access '/tmp/go-build808637166': No such file or directory > > > For the record, here are the go version and the commit hash for > gopkg.in/vmihailenco/msgpack.v2 > > $ go version > go version go1.7beta2 linux/amd64 > > $ git -C $GOPATH/src/gopkg.in/vmihailenco/msgpack.v2 rev-parse HEAD > 6915de5f21af6f5bcd2517da00d99c5131bb1da7 > > Is there a way to keep the temporary directory? I lookup the output of > "go test -h" but I couldn't find any. > > Regards, > Hiroaki > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: encoding/json: any way to receive number literal "10.0" into integer field?
https://play.golang.org/p/8R6QmYw_28 On Fri, Jun 17, 2016 at 3:08 PM Evan Digbywrote: > To further this, if you do need your internal model to be a concrete > struct rather than interface this approach still fits: > > https://play.golang.org/p/_wrgN1FltA > > > On Friday, 17 June 2016 15:02:43 UTC-7, Evan Digby wrote: >> >> One approach to deal with this would be to abstract your internal model >> as an interface from the model you receive and mutate them to match. This >> also gives you the power to truncate/round/mutate a float however best >> suites your needs rather than hoping the library truncates/rounds/mutates >> in a way that meets your requirements. >> >> https://play.golang.org/p/d97ElsNa_o >> >> You could limit your internal model to just the information you need, and >> in a format that better matches your requirements than the structure >> dictated by the message producer. >> >> There is of course overhead to this solution, but in many many (not all) >> applications this approach is worth the overhead of matching the producer's >> data format to your interface. >> >> It'll allow you to easily tack on new sources of data (if that's ever a >> requirements). If they do change their format, you don't have to change all >> of your downstream code. You just have to change the implementation of the >> SAME interface to match the new source data. >> >> On Friday, 17 June 2016 14:37:19 UTC-7, Jonathan Gold wrote: >>> >>> We have an upstream provider of JSON data which sends integer data >>> types as floating point literals with a "0" decimal part. By default >>> json.Decoder doesn't allow this and fails: >>> >>> https://play.golang.org/p/MEJlg1n3vs >>> >>> Unfortunately this provider is almost certain not to change this, and >>> so we just have to deal with it as clients of their API. What are some >>> ways we could handle this? >>> >>> Things that come to mind: >>> >>> - Submit a change to encoding/json that allows us to set some sort >>> of number processing function to customize its behavior (or to >>> set some sort of policy which allows number literals ending in >>> \.0*$ for integer targets) >>> >>> - Create a custom io.Reader which wraps in input stream by reading >>> individual tokens from it using an internal json.Decoder, and >>> then emits number literals without trailing ".0". >>> >>> - Buffer all inbound decodes into a map[string]interface{} and set >>> the decoder to UseNumber(), and then replace those json.Numbers >>> with corrected ones, then serialize back and decode once more >>> into the destination (this is similar to the token-stream >>> approach above) >>> >>> Are there other approaches I'm overlooking? >>> >>> jonathan >>> >> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
On Friday, June 17, 2016 at 4:48:06 PM UTC+2, gordo...@gmail.com wrote: > > I don't see the point to the exercise as far as optimizing golang is > > concerned. > It is a general rule that using more registers results in faster code. Yes, except when they're wasted as in using the extra register is golang 1.7 where to save a load immediate $1. > > Your experiment just shows that Your compiler (GCC?) > My post containing the example mentions that the compiler is clang 3.9. Yes, sorry, I was tired and forgot that. > > missed an optimization as far as reducing backend latency goes. > It is significant because it indicates that the clang compiler might be > completely missing the general rule which I mentioned above. It is significant to Clang development, but not to golang developement, which I thought was what we were working on in this thread; that's why I said that your work, though important, doesn't really apply here as to golang improvements. > > You may also find that swapping the order of some of the instructions such > > as the second and the third in the loop may also reduce backend latency > > further. > Swapping the order of instruction in the example results in the same or lower > performance. I said "may". > > I am not on a high end Intel CPU now, but when I was I found that with a > > buffer size adjusted to the L1 cache size (8192 32-bit words or 32 > > Kilobytes) that eratspeed ran on an Intel 2700K @ 3.5 GHz at about 3.5 > > clock cycles per loop (about 405,000,000 loops for this range). > > My current AMD Bulldozer CPU has a severe cache bottleneck and can't come > > close to this speed by a factor of about two. > Which Bulldozer version do you have: original Bulldozer, Piledriver, > Steamroller/Kaveri or Excavator/Carrizo? One of the original's, a FX8120 (4 core, 8 processor) @ 3.1 GHz. by the clock frequency, I assume you were running these tests on a high end Intel CPU? Have you tried compiling eratspeed with a new version of GCC to see how it compares to Clang? I am currently working on a golang version of eratspeed and we'll see how it compares; it will be a better comparison that just using my simple PrimesSpeed test... -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] encoding/json: any way to receive number literal "10.0" into integer field?
Thanks -- I should have clarified that we've ruled that option out (it was our first approach), since we want to stick to the built in types that won't require explicit exchange with the primitive types. jonathan On Fri, Jun 17, 2016 at 02:56:49PM -0700, Edward Muller wrote: > AFAIK (and someone will probably provide something better) you need to do > something like this: > > https://play.golang.org/p/riq-m6cgZu > > > On Jun 17, 2016, at 2:37 PM, jg...@bitgirder.com wrote: > > > > We have an upstream provider of JSON data which sends integer data > > types as floating point literals with a "0" decimal part. By default > > json.Decoder doesn't allow this and fails: > > > >https://play.golang.org/p/MEJlg1n3vs > > > > Unfortunately this provider is almost certain not to change this, and > > so we just have to deal with it as clients of their API. What are some > > ways we could handle this? > > > > Things that come to mind: > > > >- Submit a change to encoding/json that allows us to set some sort > > of number processing function to customize its behavior (or to > > set some sort of policy which allows number literals ending in > > \.0*$ for integer targets) > > > >- Create a custom io.Reader which wraps in input stream by reading > > individual tokens from it using an internal json.Decoder, and > > then emits number literals without trailing ".0". > > > >- Buffer all inbound decodes into a map[string]interface{} and set > > the decoder to UseNumber(), and then replace those json.Numbers > > with corrected ones, then serialize back and decode once more > > into the destination (this is similar to the token-stream > > approach above) > > > > Are there other approaches I'm overlooking? > > > > jonathan > > > > -- > > You received this message because you are subscribed to the Google Groups > > "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to golang-nuts+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: encoding/json: any way to receive number literal "10.0" into integer field?
One approach to deal with this would be to abstract your internal model as an interface from the model you receive and mutate them to match. This also gives you the power to truncate/round/mutate a float however best suites your needs rather than hoping the library truncates/rounds/mutates in a way that meets your requirements. https://play.golang.org/p/d97ElsNa_o You could limit your internal model to just the information you need, and in a format that better matches your requirements than the structure dictated by the message producer. There is of course overhead to this solution, but in many many (not all) applications this approach is worth the overhead of matching the producer's data format to your interface. It'll allow you to easily tack on new sources of data (if that's ever a requirements). If they do change their format, you don't have to change all of your downstream code. You just have to change the implementation of the SAME interface to match the new source data. On Friday, 17 June 2016 14:37:19 UTC-7, Jonathan Gold wrote: > > We have an upstream provider of JSON data which sends integer data > types as floating point literals with a "0" decimal part. By default > json.Decoder doesn't allow this and fails: > > https://play.golang.org/p/MEJlg1n3vs > > Unfortunately this provider is almost certain not to change this, and > so we just have to deal with it as clients of their API. What are some > ways we could handle this? > > Things that come to mind: > > - Submit a change to encoding/json that allows us to set some sort > of number processing function to customize its behavior (or to > set some sort of policy which allows number literals ending in > \.0*$ for integer targets) > > - Create a custom io.Reader which wraps in input stream by reading > individual tokens from it using an internal json.Decoder, and > then emits number literals without trailing ".0". > > - Buffer all inbound decodes into a map[string]interface{} and set > the decoder to UseNumber(), and then replace those json.Numbers > with corrected ones, then serialize back and decode once more > into the destination (this is similar to the token-stream > approach above) > > Are there other approaches I'm overlooking? > > jonathan > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] export struct method to C-code as callback (cgo-related)
If the library made no provision for the caller to give a unique token which would help to identify it in the callback then that library does not expect to have more than one caller-and-callback in play at any one time, or, even worse, expects you to arrange some thread-local storage in the caller to resolve the correct one. Without knowing more about the library there isn't much to say, except that you should be able to serialize all calls into it on the Go side, leaving only one possible *A with an outstanding function call at a time. This is going to be slow. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] encoding/json: any way to receive number literal "10.0" into integer field?
AFAIK (and someone will probably provide something better) you need to do something like this: https://play.golang.org/p/riq-m6cgZu > On Jun 17, 2016, at 2:37 PM, jg...@bitgirder.com wrote: > > We have an upstream provider of JSON data which sends integer data > types as floating point literals with a "0" decimal part. By default > json.Decoder doesn't allow this and fails: > >https://play.golang.org/p/MEJlg1n3vs > > Unfortunately this provider is almost certain not to change this, and > so we just have to deal with it as clients of their API. What are some > ways we could handle this? > > Things that come to mind: > >- Submit a change to encoding/json that allows us to set some sort > of number processing function to customize its behavior (or to > set some sort of policy which allows number literals ending in > \.0*$ for integer targets) > >- Create a custom io.Reader which wraps in input stream by reading > individual tokens from it using an internal json.Decoder, and > then emits number literals without trailing ".0". > >- Buffer all inbound decodes into a map[string]interface{} and set > the decoder to UseNumber(), and then replace those json.Numbers > with corrected ones, then serialize back and decode once more > into the destination (this is similar to the token-stream > approach above) > > Are there other approaches I'm overlooking? > > jonathan > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] encoding/json: any way to receive number literal "10.0" into integer field?
We have an upstream provider of JSON data which sends integer data types as floating point literals with a "0" decimal part. By default json.Decoder doesn't allow this and fails: https://play.golang.org/p/MEJlg1n3vs Unfortunately this provider is almost certain not to change this, and so we just have to deal with it as clients of their API. What are some ways we could handle this? Things that come to mind: - Submit a change to encoding/json that allows us to set some sort of number processing function to customize its behavior (or to set some sort of policy which allows number literals ending in \.0*$ for integer targets) - Create a custom io.Reader which wraps in input stream by reading individual tokens from it using an internal json.Decoder, and then emits number literals without trailing ".0". - Buffer all inbound decodes into a map[string]interface{} and set the decoder to UseNumber(), and then replace those json.Numbers with corrected ones, then serialize back and decode once more into the destination (this is similar to the token-stream approach above) Are there other approaches I'm overlooking? jonathan -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: importer.Default not able to find packages in $GOPATH
'go install' worked; thanks a ton! I'll also take a look at the other stuff you mentioned because I'm curious about the details. On Fri, Jun 17, 2016 at 5:15 PM, adonovan via golang-nuts < golang-nuts@googlegroups.com> wrote: > On Friday, 17 June 2016 15:21:55 UTC-4, Joshua Liebow-Feeser wrote: >> >> Hi All, >> >> I'm trying to use the go/* packages to parse and type check Go source >> code. I've downloaded github.com/coreos/etcd to test this on, and I'm >> currently trying it out in the etcdserver subdirectory. When I run 'go >> build' everything works fine, but when I try to type check, none of the >> imports are found. >> > > Try running 'go install' or 'go build -i' on the packages you want to > analyze. > > The default importer works great for packages in the standard > library, which are always available in compiled form, but poorly for user > packages, for which it may have been days or weeks since the most recent > install. Take a look at the golang.org/x/tools/go/loader package, > which loads all packages from source. > > Also, you may find this tutorial helpful: > https://github.com/golang/example/tree/master/gotypes > > -- > You received this message because you are subscribed to a topic in the > Google Groups "golang-nuts" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/golang-nuts/kJmbdFqQ0is/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: importer.Default not able to find packages in $GOPATH
On Friday, 17 June 2016 15:21:55 UTC-4, Joshua Liebow-Feeser wrote: > > Hi All, > > I'm trying to use the go/* packages to parse and type check Go source > code. I've downloaded github.com/coreos/etcd to test this on, and I'm > currently trying it out in the etcdserver subdirectory. When I run 'go > build' everything works fine, but when I try to type check, none of the > imports are found. > Try running 'go install' or 'go build -i' on the packages you want to analyze. The default importer works great for packages in the standard library, which are always available in compiled form, but poorly for user packages, for which it may have been days or weeks since the most recent install. Take a look at the golang.org/x/tools/go/loader package, which loads all packages from source. Also, you may find this tutorial helpful: https://github.com/golang/example/tree/master/gotypes -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] CPU fairness in goroutine scheuding
On Fri, Jun 17, 2016 at 2:23 PM, Dmitry Orlovwrote: > If I understand it correctly, go scheduler instruments code with additional > yield points not related to I/O. That can help fairness too. It will sometimes pre-empt on function calls, but this can get a bit hairy since you can't always be sure what function calls will be inlined. This has caused problems for me before where I was adding lots of CPU intensive goroutines to the schedulers queue, and it was context switching so much that it never finished any of them (which I suppose shouldn't have suprised me, but it would be an easy mistake for a new user who may expect CPU intensive work not to yield to make). For example, see this (not-very-scientific) analysis of some bcrypt benchmarks (TL;DR they get faster if we make inlining a lot more aggressive) which has many thousands of function calls in a loop in the blowfish block cipher: http://bl.ocks.org/SamWhited/ebe4f5923526c0d9220f1b5b23b56eba —Sam -- Sam Whited pub 4096R/54083AE104EA7AD3 https://blog.samwhited.com -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] export struct method to C-code as callback (cgo-related)
Hi Andrey! thanks for answer! But actually I don't understand how to receive context inside callback from C-code. May be I'm wrong here but as I understand you mean that I have to identify instance of *A using callback parameters received from C-side. But what about if I'll receive only sum of two arguments a+b, I don't know the result until I'll receive it from C-side and I don't know which instance should be called if callback receive ex: 10 - should it be an instance which handle 5+5 or another one used for 2+8. Thanks again! Sorry if I don't understand you correctly. On Friday, June 17, 2016 at 9:45:02 PM UTC+6, andrey mirtchovski wrote: > > I doubt that the cgo_callback method has the type the C library > desires. from https://golang.org/ref/spec#Method_declarations: > > The type of a method is the type of a function with the > receiver as first argument. > > I.e., the type of cgo_callback is func(a *A, ...). > > The solution is to have a proper C function callback with the correct > type signature expected by the C side and resolve the correct (a *A) > to call inside the callback. Something like: > > //export _callbackfunc > func _callbackfunc(fd C.int, context unsafe.Pointer) C._error_t { > a := findA(context) // resolves the correct *A based on the context > return C._error_t(a.Callback(int(fd))) > } > > On Fri, Jun 17, 2016 at 3:23 AM,> wrote: > > Hello, > > > > Could you guys please help me. > > > > I can't find how to export method function which belongs to some > go-struct > > to C-code (cgo part) as callback. I know how to export simple function > which > > not belongs to any type and use it as callback for C-function, but is it > > possibly to export method of concrete struct instance? since I need > > additional info when callback will be called from C-code. As example > what > > I'm trying to explain: > > > > // extern int goCallbackHandler(int, int); > > // > > // static int doAdd(int a, int b) { > > // return goCallbackHandler(a, b); > > // } > > > > import "C" > > > > type A struct { > > ... some data which used to process callback from C-side > > } > > > > // export cgo_callback > > func (a *A) cgo_callback(...){ <-main problem here > > } > > > > main { > >C.doAdd(...) > > } > > > > I can't modify c-side - I have only compiled library, so I don't know > how to > > identify which instance of struct should be used to properly process > C-call > > of go-side callback. > > > > Thanks, > > Andrey > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to golang-nuts...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: Golang Error - When marshelling data
On Fri, 17 Jun 2016 00:50:20 -0700 (PDT) User123wrote: [...] > When i return http response from a function should i use > *func makeCalls() (string, error, *http.Response, error) * > or > *func makeCalls() (string, error, http.Response, error) * > > For the struct i created above what should be the type of header > should it be **http.Header or **http.Header ??* Most of the code in the net/http package itself pass values of net/http.Response by pointer, so I'd do the same. On a side note, can I please ask you to refrain from using that lame HTML formatting of the Google Groups interface? In a plain mail reader which quite many people use to read mailing list your message looks something like [1]. As you could see, guessing which asterisk denotes a pointer and which is a stupid Google Group's attempt at emphasising is a bit cumbersome :-) 1. http://www.fastpic.info/image/9b5a0f4dcf -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
I don't see the point to the exercise as far as optimizing golang is concerned. Your experiment just shows that Your compiler (GCC?) missed an optimization as far as reducing backend latency goes. You may also find that swapping the order of some of the instructions such as the second and the third in the loop may also reduce backend latency further. I am not on a high end Intel CPU now, but when I was I found that with a buffer size adjusted to the L1 cache size (8192 32-bit words or 32 Kilobytes) that eratspeed ran on an Intel 2700K @ 3.5 GHz at about 3.5 clock cycles per loop (about 405,000,000 loops for this range). My current AMD Bulldozer CPU has a severe cache bottleneck and can't come close to this speed by a factor of about two. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] export struct method to C-code as callback (cgo-related)
Hello, Could you guys please help me. I can't find how to export method function which belongs to some go-struct to C-code (cgo part) as callback. I know how to export simple function which not belongs to any type and use it as callback for C-function, but is it possibly to export method of concrete struct instance? since I need additional info when callback will be called from C-code. As example what I'm trying to explain: // extern int goCallbackHandler(int, int); static int doAdd(int a, int b) {// return goCallbackHandler(a, b);// } import "C" type A struct { ... some data which used to process callback from C-side } // export cgo_callback func (a *A) cgo_callback(...){ <-main problem here } main { C.doAdd(...) } I can't modify c-side - I have only compiled library, so I don't know how to identify which instance of struct should be used to properly process C-call of go-side callback. Thanks, Andrey -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] When should I use the connection pool?
Hi to all, I'm new here and using golang too. I've read some articles and post on Connection Pooling" but I'm still confused. What I'm looking for is a rule of thumb that make me easy to decide when use connection pool. My scenario isn't so complicated: - a web application that makes CRUD operations on a Postgres database. - Minimun 10 clients connected up to 30. Moreover, I'm going to use pgx driver. I'd be grateful if you could help me to get the difference from use or not the connection pool, which is the best practice? Thanks in advance, Danilo -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Gomobile: Large .so/aar file sizes on Android
Hi All I am using Go version 1.6.2 and Gomobile (sha +6b7a416 from the tip) and to build an Android library. Our code is mostly networking "encoding/json" "fmt" "log" "math/rand" "net" "sort" "time" -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
On Friday, June 17, 2016 at 3:52:27 PM UTC+2, gordo...@gmail.com wrote: > > I don't see the point of the exercise other than it proves that not > putting the result of an operation back into the same register reduces the > latency slightly for your processor (whatever it is?); I suspect that if > you used any other register such as the unused AX register rather then the > R11 register, the results would be the same. > Why do you see no point of the exercise and at the same time you do see the point of optimizing 1.7beta1 code? What is the difference? What would be interesting is to compare the run time on your processor with > the same eratspeed program written in golang using version 1.7beta1 with > and without array bounds checks. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
I don't see the point of the exercise other than it proves that not putting the result of an operation back into the same register reduces the latency slightly for your processor (whatever it is?); I suspect that if you used any other register such as the unused AX register rather then the R11 register, the results would be the same. What would be interesting is to compare the run time on your processor with the same eratspeed program written in golang using version 1.7beta1 with and without array bounds checks. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
On Friday, June 17, 2016 at 2:30:12 AM UTC+2, gordo...@gmail.com wrote: > > > Modern x86 CPUs don't work like that. > > > In general, optimally scheduled assembly code which uses more registers > has higher performance than optimally scheduled assembly code which uses > smaller number of registers. Assuming both assembly codes correspond to the > same source code. > > > Register renaming: since Intel Pentium Pro and AMD K5. > > > Suggestion for reading: > http://www.agner.org/optimize/microarchitecture.pdf > > > An excerpt from the above PDF document (Section 10 about Haswell and > Broadwell pipeline): "... the register file has 168 integer registers and > 168 vector registers ..." > > I am aware of all of the above and have already read Agner Fogg's > publications. In addition modern CPU's do Out of Order Execution (OOE) so > rearrange the instructions to best reduce instruction latencies and > increase throughput given that there are parallel execution pipelines and > ahead-of-time execution, so the actual execution order is almost certainly > not as per the assembly listing. > > Yes, both assembly listings are from the same tight loop code, but the > "C/C++" one has been converted from another assembly format to the golang > assembly format. > > Daniel Bernstein, the author of "primegen" wrote for the Pentium 3 in x86 > (32-bit) code, as the Pentium Pro processor wasn't commonly available at > that time and 64-bit code didn't exist. His hand optimized C code for the > Sieve of Eratosthenes ("eratspeed.c" in the "doit()" function for the > "while k < B loop") uses six registers for this inner culling loop being > discussed, and takes about 3.5 CPU clock cycles per loop on a modern CPU > (Haswell). > The following isn't an argument against your post or in favor of your post. It is just some assembly code that I find interesting. $ clang-3.9 -S -O3 eratspeed.c -mtune=bdver3 .LBB1_4: movl%edx, %edi shrl$5, %edi movl%edx, %ecx andl$31, %ecx movltwo(,%rcx,4), *%ecx* addl%esi, %edx orl *%ecx*, a(%rbx,%rdi,4) cmpl$32032, %edx jb .LBB1_4 5 registers: DX DI CX SI BX $ perf stat --repeat=100 -- ./eratspeed.orig >/dev/null Performance counter stats for './eratspeed.orig' (100 runs): 466.592220 task-clock (msec) #0.999 CPUs utilized ( +- 0.11% ) 1 context-switches #0.002 K/sec ( +- 12.37% ) 0 cpu-migrations#0.000 K/sec ( +-100.00% ) 86 page-faults #0.185 K/sec ( +- 0.09% ) 1,862,076,369 cycles#3.991 GHz ( +- 0.11% ) 74,805,707 stalled-cycles-frontend #4.02% frontend cycles idle ( +- 1.20% ) 272,721,373 stalled-cycles-backend# *14.65%* backend cycles idle ( +- 0.16% ) 4,116,301,949 instructions #*2.21* insn per cycle #0.07 stalled cycles per insn ( +- 0.00% ) 473,019,237 branches # 1013.774 M/sec ( +- 0.00% ) 8,443,283 branch-misses #1.78% of all branches ( +- 1.05% ) 0.467167554 seconds time elapsed ( +- 0.11% ) - Hand optimization of the above: .LBB1_4: movl%edx, %edi shrl$5, %edi movl%edx, %ecx andl$31, %ecx movltwo(,%rcx,4), *%r11d* addl%esi, %edx orl *%r11d*, a(%rbx,%rdi,4) cmpl$32032, %edx jb .LBB1_4 6 registers: DX DI CX SI BX R11 $ perf stat --repeat=100 -- ./eratspeed.opt >/dev/null Performance counter stats for './eratspeed.opt' (100 runs): 444.740487 task-clock (msec) #0.999 CPUs utilized ( +- 0.01% ) 1 context-switches #0.002 K/sec ( +- 11.68% ) 0 cpu-migrations#0.000 K/sec ( +-100.00% ) 85 page-faults #0.191 K/sec ( +- 0.10% ) 1,775,283,795 cycles#3.992 GHz ( +- 0.00% ) 68,397,472 stalled-cycles-frontend #3.85% frontend cycles idle ( +- 0.04% ) 201,687,390 stalled-cycles-backend# *11.36%* backend cycles idle ( +- 0.02% ) 4,116,277,783 instructions #*2.32* insn per cycle #0.05 stalled cycles per insn ( +- 0.00% ) 473,014,763 branches # 1063.575 M/sec
[go-nuts] Re: Go 1.7 Beta 2 is released
Thanks, is there a way to know what to expect in later versions ( 1.8,1.9,1.10 and so forth ... ) ? Le vendredi 17 juin 2016 01:15:18 UTC+2, Chris Broadfoot a écrit : > > Hello gophers, > > We have just released go1.7beta2, a beta version of Go 1.7. > It is cut from the master branch at the revision tagged go1.7beta2. > > Please help us by testing your Go programs with the release, and report > any problems using the issue tracker: > https://golang.org/issue/new > > You can download binary and source distributions from the usual place: > https://golang.org/dl/#go1.7beta2 > > To find out what has changed in Go 1.7, read the draft release notes: > https://tip.golang.org/doc/go1.7 > > Documentation for Go 1.7 is available at: > https://tip.golang.org/ > > Our goal is to release the final version of Go 1.7 on the 1st of August. > > Cheers, > Chris > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: How to manage a web portal with multiple services without stopping and restarting everything at each release/fix?
Hi Tamás, thanks, I will have a look. I guess and hope that this is what I need. @Simon: thanks for the response with so many details. I have been working for years for projects with load balancers, HA, disaster recovery sites, and so on so your detailed answer can be really helpful to understand the topic for other readers. In my specific case it's not what I was looking for because I am more interested in a modular software architecture that allows me to make modifications in that way, rather than the service one. Thanks. I guess that I will build up my special golang architecture and I hope that the graceful restart pattern is what I need. Cheers! Romano -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Testing best practice: passing and failing test
By the way, Go 1.7 support subtests : https://tip.golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks Which is one of the nicest additions to Go 1.7 Le jeudi 16 juin 2016 16:04:11 UTC+2, Rayland a écrit : > > Let's say I have some piece of code like this: > > type Foo struct { > } > > func (this *Foo) Do() { > } > > > If I want to test that method Foo will pass in a certain scenario and fail > in another scenario should I have a test for each scenario or should I have > all scenarios tested in a method TestFoo_Do()? > > > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Currying in Go
What is the point of doing that in a language that doesn't support auto currying ? There is none. You are not currying anything by the way, you just wrote 3 closures. Le vendredi 17 juin 2016 00:00:43 UTC+2, Zauberkraut a écrit : > > Hello, > > Go enables the evaluation of functions using currying over function > literals. Every example I've found of this is rather shallow; a "deeper" > example I wrote implementing (x => (y => (z => x^2 + y^2 + z^2))) follows: > > func f(x int) func(int) func(int) int { > return func(y int) func(int) int { > return func(z int) int { > return x*x + y*y + z*z > } > } > } > > Go's limited type inference makes the explicit, cascading function types > necessary; this seems like an eyesore and maintenance concern. While I > don't really mind it, it does cause me to hear code review sirens going off > in the distance. Generally speaking, would an extended usage of this > paradigm be considered unidiomatic in Go? Obviously, the above example is > contrived and not the sort of use case in question. Thanks! > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: polymorphism (for Go 2.0), new fast playground is live
On Thursday, June 16, 2016 at 3:20:10 PM UTC+2, Sean Russell wrote: > > I'm sorry -- the "share" button doesn't seem to be working, or doesn't > work in Firefox or Chrome. And I have a window hung on "waiting for remote > server," so I might have broken your playground. > TL:DR type conversion from arbitrary type to another arbitrary type is not directly possible with my proposal and needs to be done in two passes via a common type. This is a tough task, and won't work directly using single parametric polymorphism the way you propose. Type conversion between two arbitrary types could become the single most significant weak spot of single parametric polymorphism and a huge argument for multi parametric . > Here's my code in (hopefully) compileable form for your processor: > > package main > > impoort "strconv" > > // Given: > func map(f(*), a []*, result []*) { >for i, k := range a { > result[i] = f(k) >} > } > > This function would compile, but would not do what you want. map() would not convert arbitrary types. It would simply duplicate slices. map() would not be callable with different types of *a* and *result, * > func main() { >a := "a" ; aa := "aa"; aaa = "aaa" >mylen := func(k *string) *int { > rv := len(*k) > return >} >map(mylen, []*string{, , }, make([]*int, 3)) // Is this > possible? > >one := 1; two := 2; three := 3 >myItoA := func(k *int) *string { > rv := strconv.Itoa(*k) > return >} >map(myItoa, []*int{,,}, make([]*string, 3)) // once > more, for profit > >map(strconv.Itoa, []*int{}, make([]*float, 1)) // Is this a > runtime error? > } > The way to solve this example (slice type conversion), while keeping the style, would be to do it in two passes: First pass: convert []*anything to []interface{} Second pass: convert []interface{} to []*something_else //First pass just wraps the values to interface{}: func wrapvalues(in []*, out *[]interface{}) { for i:=0; i < len(in), i++ { *out = append(*out, in[i]) } } //Second pass does the heavy lifting and conversion: func toslice(convert func(*,interface{}), in *[]interface{}, out []*) { for i:=0; i < len(in), i++ { convert( out[i], in[i]) } } // conversion function from *int to *string func int_to_str(in interace{}, out *string) { var i = in.(int) *out = strconv.Itoa(i) } // called like this var originalslice []*int var intermediate []interface{} var result []*string = make( []*string, len(originalslice)) wrapvalues(originalslice, ) toslice(int_to_str, , result) Other solutions: An one-liner, simply traverse the original slice, and convert elements on the go: func foreach_slice(slice []*, call func(*)) { for i := range slice { call(slice[i]) } } foreach_slice(as, func(arg *string){var i int; i = len(*arg); bs = append(bs,)}) http://tsoh.host/test.html#075b8a23 Reusable conversion routine, is able to convert arbitrary slice to another arbitrary slice. Each element is converted via string and then to the target type Routine can be called using one-liner and is fully type safe: slice_via_str_from(as, my_float64_to_string).to(bs, my_string_to_int) http://tsoh.host/test.html#eaa6317e > > And, yeah... the pointers are a bit of trouble. By the time all of the > wrapping and referencing is done, it's only a little easier than just using > interface{} everywhere with casts. If you don't get type safety with this, > I feel like the syntactic sugar value is pretty modest. > > --- SER > I feel that the type safety is preserved -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: How to manage a web portal with multiple services without stopping and restarting everything at each release/fix?
This is not a question about Go, but basic web infrastructure. However, it's one of those "best practice" questions which "everybody knows the answer to" and so you may be hard put to find out more without knowing where to start. First of all, I should point out that when you tweak your PHP page, that does cause a service outage, but it's very short, and you usually don't notice. The usual solution to this problem is to use a team of web servers behind a load balancer or a reverse proxy. A load balancer is a commercial product and most router manufacturers supply them, Cisco to name but one. You can build your own reverse proxy using the Apache web server. The Apache documentation explains how. A reverse proxy is a web server that stands between the client web browsers and a team of web servers that do the work. The proxy takes each incoming request and sends it to one of the web servers, then relays the response back to the client. it looks to the client that the proxy is its web server, but actually it's just relaying requests and responses. Add the ability to put extra worker servers into the team and take them out and you have your solution. When you want to restart a server, you remove it from the team, restart it and then put it back, so at no time are all the servers out of action. You need to arrange that there are always enough servers to cope with the traffic. If you visit a large website such Google or Amazon, you are almost certainly talking to some sort of reverse proxy with lots of workers behind it. The worker servers are usually called "application servers" , so you have the classic chain of: load balancer, web server, application server. In your case, your Go software will run on the app servers. Access to the app servers should be disallowed by the network except by the web server. One of the advantages of the REST approach to web design is that your servers can be stateless, so each request can be sent to any old server. (With a statefull protocol, the proxy may need to keep track of sessions of associated requests and send all of the request in the session to the same worker.) It's also common to terminate any SSL connections at the web server, ie the clients communicate with the web server via HTTPS, but it communicates with the app servers via HTTP. Being encrypted, HTTPS requires significant extra work and Apache, in particular, is very good at this. A load balancer is a reverse proxy with lots of extra features. In particular it may have more than one connection to the internet, so if there is a problem with one of the connections, it will still work. Load balancers themselves can be more than one machine working as a team, possibly on different sites. By spending (a lot) more money you can increase the reliability of your web service by having equipment in separate buildings on separate sites with a distributed load balancer network, Internet connections from different providers coming into different parts of the buildings and arriving at your equipment via different physical routes, multiple redundant uninterruptable power supplies, backup diesel generators, and so on. One organisation I worked for ran two buildings, one on each side of London, both staffed and each with enough equipment to provide the web service single-handed. The thinking was that there was small chance of one building being destroyed by war, flood, fire or a plane crash. They switched the production service between the buildings regularly to make sure that everything still worked. This is not a cheap solution, but they were in the banking sector and they could afford it. On the more affordable side, cloud services often provide a load balancer as part of the mix. This is general web infrastructure wisdom and nothing to do with Go, so let's not let this discussion run on. There should be enough above to help you choose the right Google searches. Unless anybody thinks that what I've written contains any major blunders, let's all get back to talking about Go. (If anybody wants to contact me directly for help with this sort of stuff, I'm a contractor and my daily rate is only slightly heinous.) Hoping that this helps. Simon On Thursday, June 16, 2016 at 4:02:50 PM UTC+1, romano.p...@gmail.com wrote: > > Hi All, > > Forgive me if I posted in the wrong group, as it could be more of a > question related to design patterns rather than to golang itself. > > *The context* > Imagine to create a full portal with users, profiles, ... and multiple > services that can be activated for different customers. > I can organise the code of my portal in modules so to have a "quite" clear > use of golang packages, but at the end of the day I get to a unique EXE > file that represents my server/portal. > > When I start it, the portal runs. Perfect. > > To make an example, think of google, where you have Drive (Service 1) and > Gmail
Re: [go-nuts] Re: [ANN] primegen.go: Sieve of Atkin prime number generator
> Looks like something is wrong with immediate loading for the 1 << ... > operation. Could you open a bug with repro instructions? I can look at it > when 1.8 opens. I have opened a golang issue #16092 as follows: https://github.com/golang/go/issues/16092 I may have over-complicated it, but as well as the failure to use the load immediate instruction, I documented other suggestions that would make the code run faster as well as a method to eliminate the array bounds check for cases when the check is built-in to the loop as C# (sometimes) does for x86 code when one uses a specific formulation of a loop. If able to be implemented, these changes would then apply to x86 code as well due to reduction of register use, and could potentially make golang code run as fast as C/C++ as compared in the issue. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: map memory usage question
On Friday, 17 June 2016 02:59:49 UTC+3, kortschak wrote: > > I'm running a terabyte-scale (minor compiler changes are necessary to get > this to run) genome resequencing simulation at the moment and an > interesting question has arisen. > The simulation involved hashing over ~all positions of a genome, either > with the key being a string or a [2]string. > Interesting :) Note there might be better data-structures for this than the built-in map. Do you have the code uploaded with some example which uses under 8GB of memory? Or a description how the algorithm works? The difference in the resident memory size for the entire program for these > two cases is about a factor of two. > > What is an estimate of the contribution of key size to the number of bytes > required per element for a map? > -- > afk > Dan > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Golang Error - When marshelling data
Have found the cause of the issue The message that I was trying to marshall had *0x10a3c2d0 (when fmt.Println) *which was http response. >From https://github.com/revel/revel/issues/1037 i found that *Since Go 1.6 http.Request struct contains `Cancel <-chan struct{}` which* *results in `json: unsupported type: <-chan struct {}`* So i created struct for httpRes instaed of using interface . *type HttpRespStruct struct {* * Status string* * StatusCode int* * Proto string* * Header http.Header * * ContentLength int64* * TransferEncoding []string* *}* and it fixed the issue. One thing i wanted to ask When i return http response from a function should i use *func makeCalls() (string, error, *http.Response, error) * or *func makeCalls() (string, error, http.Response, error) * For the struct i created above what should be the type of header should it be **http.Header or **http.Header ??* Thank You. On Thursday, June 16, 2016 at 3:14:00 PM UTC+5:30, User123 wrote: > The data I am trying to marshal contains serialized data. This data is > received in http response. > > It works perfectly when I try in version 1.4.1/ 1.4.2. > > But when I try this to the latest version 1.6 it gives an error: *json: > unsupported type: <-chan struct {}* > > Why is it so? > > I am not able to upgrade to the latest version because of this. > > Please help! > > > > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.