Re: [go-nuts] types of slice indices and range operators

[Robert Engels]

Also, what you are asking for I believe is covered by generics. 

> On Feb 2, 2020, at 8:17 PM, Steve Roth  wrote:
> 
> 
> Oh, please, Robert.  No need to be condescending.  I know perfectly well how 
> to use the value when that's what I want.  Often it is not.  The examples 
> that I cited are such a case: we are adding the IDs of the events — the 
> indices of the slice — to the person.Events field.  If you don't like my 
> example, take a look through the standard library code: you'll find a great 
> many instances of range loops using only the index.
> 
> Regards,
> Steve
> 
> 
>> On Sun, Feb 2, 2020 at 5:56 PM Robert Engels  wrote:
>> You are using range incorrectly - you are using the index not the value. 
>> 
>> See https://tour.golang.org/moretypes/16
>> 
>>>> On Feb 2, 2020, at 7:39 PM, Steve Roth  wrote:
>>>> 
>>> 
>>> Greetings,
>>> 
>>> I'm considering submitting a proposal for a language change, and would like 
>>> some informal feedback on the idea before engaging the formal process.  The 
>>> idea being proposed here is intended to improve type safety by removing the 
>>> need for some error-prone type casting.  It is a backward compatible change.
>>> 
>>> Consider a body of code that works with people and events.  Both are 
>>> identified with integer IDs; in order to prevent them being used in the 
>>> wrong contexts, they are given specific integer types:
>>> type EventID int
>>> type PersonID int
>>> Assume also that people have lists of events they attend:
>>> type Person struct {
>>> Events []EventID
>>> }
>>> The code maintains slices of each type:
>>> var events []*Event
>>> var people []*Person
>>> When indexing into those slices, one can use the appropriate ID type:
>>> var eventID EventID = 3
>>> println(events[eventID])
>>> However, iterating over these slices requires casting in the current Go 
>>> specification:
>>> for eventID := range events {
>>> // eventID here has type int, not type EventID
>>> person.Events = append(person.Events, eventID) // compile error, wrong type
>>> person.Events = append(person.Events, EventID(eventID)) // cast required
>>> }
>>> In cases where the event ID needs to be used inside the loop, it has lost 
>>> its type safety.  It has to be casted back to the type it is supposed to 
>>> have.  And that casting is error-prone; I could easily cast it to PersonID 
>>> by mistake.  This seems to be a noteworthy gap in type safety.
>>> 
>>> My proposal is to allow this construction:
>>> var eventID EventID
>>> for eventID = range events {
>>> // eventID here has type EventID
>>> person.Events = append(person.Events, eventID) // accepted by compiler
>>> }
>>> Phrased more formally: a slice range operator can be assigned to a variable 
>>> of any integer-derived type, not just "int".  If this were done, 
>>> error-prone casting could be avoided.
>>> 
>>> Admittedly, there is still room for error here, since it would be possible 
>>> to write
>>> var eventID PersonID
>>> for eventID = range events {
>>> // eventID here has the wrong type PersonID
>>> person.Events = append(person.Events, eventID) // compile error, wrong type
>>> }
>>> However, this error seems far less likely that a mistaken cast.  And since 
>>> there's no expectation that casts should be needed, the compiler error 
>>> would cause greater scrutiny leading to fixing the real problem.  (In any 
>>> case, I don't wish to propose the (much larger and more impactful) change 
>>> of having slices with typed indices.)
>>> 
>>> I believe this proposal would significantly improve type safety in programs 
>>> that work with multiple integer-derived types (such as most anything 
>>> working with a relational database that prefers integer primary keys).  I 
>>> also believe it is backward compatible since it does not change the 
>>> semantic of any existing code; it just adds a semantic to code that 
>>> currently would not compile.
>>> 
>>> I solicit the community's feedback.  Should this proposal be formally 
>>> submitted?
>>> 
>>> Regards,
>>> Steve
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe

Re: [go-nuts] types of slice indices and range operators

[Robert Engels]

It not being condescending. Your code is not correct. It is not an eventID - it 
is an index (an int). Slice indexes are ints. If you want a typed key (other 
than int) you need a map (or variant). 

> On Feb 2, 2020, at 8:17 PM, Steve Roth  wrote:
> 
> 
> Oh, please, Robert.  No need to be condescending.  I know perfectly well how 
> to use the value when that's what I want.  Often it is not.  The examples 
> that I cited are such a case: we are adding the IDs of the events — the 
> indices of the slice — to the person.Events field.  If you don't like my 
> example, take a look through the standard library code: you'll find a great 
> many instances of range loops using only the index.
> 
> Regards,
> Steve
> 
> 
>> On Sun, Feb 2, 2020 at 5:56 PM Robert Engels  wrote:
>> You are using range incorrectly - you are using the index not the value. 
>> 
>> See https://tour.golang.org/moretypes/16
>> 
>>>> On Feb 2, 2020, at 7:39 PM, Steve Roth  wrote:
>>>> 
>>> 
>>> Greetings,
>>> 
>>> I'm considering submitting a proposal for a language change, and would like 
>>> some informal feedback on the idea before engaging the formal process.  The 
>>> idea being proposed here is intended to improve type safety by removing the 
>>> need for some error-prone type casting.  It is a backward compatible change.
>>> 
>>> Consider a body of code that works with people and events.  Both are 
>>> identified with integer IDs; in order to prevent them being used in the 
>>> wrong contexts, they are given specific integer types:
>>> type EventID int
>>> type PersonID int
>>> Assume also that people have lists of events they attend:
>>> type Person struct {
>>> Events []EventID
>>> }
>>> The code maintains slices of each type:
>>> var events []*Event
>>> var people []*Person
>>> When indexing into those slices, one can use the appropriate ID type:
>>> var eventID EventID = 3
>>> println(events[eventID])
>>> However, iterating over these slices requires casting in the current Go 
>>> specification:
>>> for eventID := range events {
>>> // eventID here has type int, not type EventID
>>> person.Events = append(person.Events, eventID) // compile error, wrong type
>>> person.Events = append(person.Events, EventID(eventID)) // cast required
>>> }
>>> In cases where the event ID needs to be used inside the loop, it has lost 
>>> its type safety.  It has to be casted back to the type it is supposed to 
>>> have.  And that casting is error-prone; I could easily cast it to PersonID 
>>> by mistake.  This seems to be a noteworthy gap in type safety.
>>> 
>>> My proposal is to allow this construction:
>>> var eventID EventID
>>> for eventID = range events {
>>> // eventID here has type EventID
>>> person.Events = append(person.Events, eventID) // accepted by compiler
>>> }
>>> Phrased more formally: a slice range operator can be assigned to a variable 
>>> of any integer-derived type, not just "int".  If this were done, 
>>> error-prone casting could be avoided.
>>> 
>>> Admittedly, there is still room for error here, since it would be possible 
>>> to write
>>> var eventID PersonID
>>> for eventID = range events {
>>> // eventID here has the wrong type PersonID
>>> person.Events = append(person.Events, eventID) // compile error, wrong type
>>> }
>>> However, this error seems far less likely that a mistaken cast.  And since 
>>> there's no expectation that casts should be needed, the compiler error 
>>> would cause greater scrutiny leading to fixing the real problem.  (In any 
>>> case, I don't wish to propose the (much larger and more impactful) change 
>>> of having slices with typed indices.)
>>> 
>>> I believe this proposal would significantly improve type safety in programs 
>>> that work with multiple integer-derived types (such as most anything 
>>> working with a relational database that prefers integer primary keys).  I 
>>> also believe it is backward compatible since it does not change the 
>>> semantic of any existing code; it just adds a semantic to code that 
>>> currently would not compile.
>>> 
>>> I solicit the community's feedback.  Should this proposal be formally 
>>> submitted?
>>> 
>>> Regards,
>>> Steve
>>> 
>>> -- 
>>> You received 

Re: [go-nuts] types of slice indices and range operators

[Robert Engels]

You are using range incorrectly - you are using the index not the value. 

See https://tour.golang.org/moretypes/16

> On Feb 2, 2020, at 7:39 PM, Steve Roth  wrote:
> 
> 
> Greetings,
> 
> I'm considering submitting a proposal for a language change, and would like 
> some informal feedback on the idea before engaging the formal process.  The 
> idea being proposed here is intended to improve type safety by removing the 
> need for some error-prone type casting.  It is a backward compatible change.
> 
> Consider a body of code that works with people and events.  Both are 
> identified with integer IDs; in order to prevent them being used in the wrong 
> contexts, they are given specific integer types:
> type EventID int
> type PersonID int
> Assume also that people have lists of events they attend:
> type Person struct {
> Events []EventID
> }
> The code maintains slices of each type:
> var events []*Event
> var people []*Person
> When indexing into those slices, one can use the appropriate ID type:
> var eventID EventID = 3
> println(events[eventID])
> However, iterating over these slices requires casting in the current Go 
> specification:
> for eventID := range events {
> // eventID here has type int, not type EventID
> person.Events = append(person.Events, eventID) // compile error, wrong type
> person.Events = append(person.Events, EventID(eventID)) // cast required
> }
> In cases where the event ID needs to be used inside the loop, it has lost its 
> type safety.  It has to be casted back to the type it is supposed to have.  
> And that casting is error-prone; I could easily cast it to PersonID by 
> mistake.  This seems to be a noteworthy gap in type safety.
> 
> My proposal is to allow this construction:
> var eventID EventID
> for eventID = range events {
> // eventID here has type EventID
> person.Events = append(person.Events, eventID) // accepted by compiler
> }
> Phrased more formally: a slice range operator can be assigned to a variable 
> of any integer-derived type, not just "int".  If this were done, error-prone 
> casting could be avoided.
> 
> Admittedly, there is still room for error here, since it would be possible to 
> write
> var eventID PersonID
> for eventID = range events {
> // eventID here has the wrong type PersonID
> person.Events = append(person.Events, eventID) // compile error, wrong type
> }
> However, this error seems far less likely that a mistaken cast.  And since 
> there's no expectation that casts should be needed, the compiler error would 
> cause greater scrutiny leading to fixing the real problem.  (In any case, I 
> don't wish to propose the (much larger and more impactful) change of having 
> slices with typed indices.)
> 
> I believe this proposal would significantly improve type safety in programs 
> that work with multiple integer-derived types (such as most anything working 
> with a relational database that prefers integer primary keys).  I also 
> believe it is backward compatible since it does not change the semantic of 
> any existing code; it just adds a semantic to code that currently would not 
> compile.
> 
> I solicit the community's feedback.  Should this proposal be formally 
> submitted?
> 
> Regards,
> Steve
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAAnpqKHS%3D2z0ZbNVSrULXLLGz%3DhAqpmrsLieciFu8L6%3DAn%3DLHA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/97C56113-BB0A-432A-B44A-F9D7F03FEEFA%40ix.netcom.com.

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Robert Engels]

fixed only supports 8 decimals places - fixed. Without digging into the 
algorithm I’m sure that is the source of your issue. 


> On Feb 1, 2020, at 4:03 PM, craterm...@gmail.com wrote:
> 
> 
> Perhaps I'm doing something wrong or using the library outside of its 
> intended purpose, but I found that this library doesn't handle Muller's 
> Recurrence correctly. For those not familiar, Muller's Recurrence is 108 - 
> (815-1500/z)/y
> 
> https://play.golang.org/p/sePTgjZzHeY
> 
> See https://latkin.org/blog/2014/11/22/mullers-recurrence-roundoff-gone-wrong/
> 
>> On Sunday, January 26, 2020 at 8:46:16 AM UTC-8, Robert Engels wrote:
>> Which is exactly what github.com/robaho/fixed and many others do!
>> 
>>>> On Jan 26, 2020, at 10:34 AM, Michael Jones  wrote:
>>>> 
>>> 
>>> ...thus the virtue of scaled integers. scaling by 100 makes cents whole, 
>>> scaling by 1*100 gives four decimal places beyond that. There is 
>>> nothing bad about floating point despite the reputation, it's just not the 
>>> number system from algebra; nor is binary floating point the same as 
>>> decimal floating point. The problems all start with false presumptions.
>>> 
>>>> On Sun, Jan 26, 2020 at 8:20 AM Robert Engels  wrote:
>>>> Just an FYI, often that is not correct. Many financial systems require 
>>>> fractional pennies due to the volume of transactions. Think about taxing 
>>>> stock exchanges the pennies add up quickly at any tax rate, so they 
>>>> use fractional pennies to reduce the size of the error bucket. 
>>>> 
>>>>>> On Jan 26, 2020, at 8:50 AM, Pat Farrell  wrote:
>>>>>> 
>>>>> 
>>>>> never use floating point if you are trying to represent money, say 
>>>>> dollars and cents or decimal values of the euro.
>>>>> Store the money as integer number of pennies.
>>>>> -- 
>>>>> You received this message because you are subscribed to the Google Groups 
>>>>> "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>>>> email to golan...@googlegroups.com.
>>>>> To view this discussion on the web visit 
>>>>> https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com.
>>>> 
>>>> -- 
>>>> You received this message because you are subscribed to the Google Groups 
>>>> "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>>> email to golan...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com.
>>> 
>>> 
>>> -- 
>>> Michael T. Jones
>>> michae...@gmail.com
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/46cadc14-9c8e-4c3a-9c6b-d0af7b621061%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F99914BF-7D69-4CEF-9C00-B583615B44AD%40ix.netcom.com.

Re: [go-nuts] Go Tool to convert svg to png/jpg

[robert engels]

That rasterx is pretty sweet. Should be pretty easy to create an SVG to PNG 
module using that.

> On Feb 1, 2020, at 12:20 AM, Randall O'Reilly  wrote:
> 
> Here’s some partial SVG renderers in Go:
> 
> * https://github.com/srwiley/oksvg
> * https://github.com/goki/gi (uses srwiley’s rasterx rasteriser, has separate 
> SVG parsing / painting impl).
> 
> Cheers,
> - Randy
> 
>> On Jan 31, 2020, at 6:54 PM, Michael Jones  wrote:
>> 
>> Just to be clear: PNG is a description of pixel values comprising an image 
>> (RGBa, RGBa, RGBa, ...), SVG is a program for creating an image (set color 
>> to blue, draw a circle, change to red, draw a line, ...). Going from SVG 
>> (scalable vector graphics) to pixels is to render an image by executing the 
>> simple program of graphical operation codes. (As said above)
>> 
>> Solutions depending on situation: most every browser has code to rasterize 
>> SVG. Can you invoke one to produce your image? There are numerous free tools 
>> to do the same--can you bundle one of those and invoke it as an external 
>> program? Cairo can do it. (these are in addition to what you really seem to 
>> want, which is a fully Go SVG -> PNG renderer.)
>> 
>> Good luck in your project,
>> michael
>> 
>> On Fri, Jan 31, 2020 at 6:03 PM robert engels  wrote:
>> There is no cross-platform graphics library included in Go.
>> 
>> Most likely you’ll need a C binding to Qt or similar to perform the 
>> rasterization.
>> 
>> You might be able to put one together using something like 
>> https://github.com/hajimehoshi/ebiten
>> 
>> 
>>> On Jan 31, 2020, at 7:31 PM, maithri.fri...@gmail.com wrote:
>>> 
>>> I'm looking for one too but can't find anything online either..
>>> 
>>> On Monday, March 9, 2015 at 11:35:51 PM UTC-7, will wrote:
>>> Hi Gophers,
>>> 
>>> Is there a Golang based tool (or library) tool to convert svg to png or jpg?
>>> 
>>> regards,
>>> 
>>> Will
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/d03ad514-1e56-42a3-b7c6-798346a76ca1%40googlegroups.com.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/BF136C90-3061-4E01-890B-A6FB38E94E07%40ix.netcom.com.
>> 
>> 
>> -- 
>> Michael T. Jones
>> michael.jo...@gmail.com
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CALoEmQw_8%3D3AkkrmZNG9Xrzq37TaauFF77wJUGjkp-Mx3%3DrUPQ%40mail.gmail.com.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/286D10E9-29D9-409F-916D-729EE4490164%40ix.netcom.com.

Re: [go-nuts] Go Tool to convert svg to png/jpg

[robert engels]

There is no cross-platform graphics library included in Go.

Most likely you’ll need a C binding to Qt or similar to perform the 
rasterization.

You might be able to put one together using something like 
https://github.com/hajimehoshi/ebiten


> On Jan 31, 2020, at 7:31 PM, maithri.fri...@gmail.com wrote:
> 
> I'm looking for one too but can't find anything online either..
> 
> On Monday, March 9, 2015 at 11:35:51 PM UTC-7, will wrote:
> Hi Gophers,
> 
> Is there a Golang based tool (or library) tool to convert svg to png or jpg?
> 
> regards,
> 
> Will
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d03ad514-1e56-42a3-b7c6-798346a76ca1%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/BF136C90-3061-4E01-890B-A6FB38E94E07%40ix.netcom.com.

Re: [go-nuts] Is it considered to separate runtime and cmd/compile irrelevant standard libraries out from the core language repo and hosted in a different module, eg. std?

[robert engels]

What I am referring to is a Go shared runtime library, that my Go executable 
would link against at runtime, so that if I compile my Go program against 
1.13.0 it will run against any 1.13.x stdlib Go runtime installed by the client.

This way for security patches, only new runtimes need to be shipped, and 
executables do not need to be recompiled. It also results in far smaller Go 
binaries (especially with greater Go adoption).

If stdlib runtime changes result in an incompatible API, the minor version 
needs to be changed, e.g. 1.13.x becomes 1.14.0

Based on this link in the FAQ Why is my trivial program such a large binary? 
<https://golang.org/doc/faq#Why_is_my_trivial_program_such_a_large_binary> I do 
not think something like this exists.


> On Jan 27, 2020, at 3:33 PM, Ian Lance Taylor  wrote:
> 
> On Mon, Jan 27, 2020 at 1:27 PM Robert Engels  <mailto:reng...@ix.netcom.com>> wrote:
>> 
>> Doesn’t this piggyback on making the runtime a shared library so it can be 
>> updated without application recompile?
> 
> I guess I'm not sure what you mean.  That already works today, for
> people who choose to use it.  It seems independent of how we manage Go
> releases.
> 
> Ian
> 
>>> On Jan 27, 2020, at 2:24 PM, Ian Lance Taylor  wrote:
>>> 
>>> On Mon, Jan 27, 2020 at 7:02 AM 'Axel Wagner' via golang-nuts
>>>  wrote:
>>>> 
>>>> The original message mentions size. The given list is 25MB/337MB 
>>>> uncompressed or 7MB/115MB compressed. So in terms of saved space, we are 
>>>> talking ~6%. It's not nothing, but it's also not a lot. Especially as 
>>>> you'll most likely need to download them right after anyway.
>>>> 
>>>> You also mentions splitting up the issue tracker. But note, that even 
>>>> though the various x-repos are already separate repositories, they still 
>>>> use the same issue-tracker. Clearly, having a unified issue tracker is 
>>>> perceived as a *benefit* to the Go team, not a detriment.
>>>> 
>>>> It might indeed be worth versioning the stdlib as modules at some point. I 
>>>> know that was discussed as part of the question of what "Go 2" means. 
>>>> Having the ability to bump major versions of the stdlib would enable to 
>>>> overhaul APIs at some point in the future. But note that the Go 1 
>>>> stability promise stays in place, so again, neither in terms of size, nor 
>>>> in terms of splitting management, this would provide any benefits. 
>>>> Programs using Go v1 stdlib packages will likely continue to exist into 
>>>> the far future, so even *if* there's at some point a v2 of the stdlib, v1 
>>>> will still need to be shipped and supported (though likely the v1 API 
>>>> would wrap v2 packages, so the actual size increase would be moderate). 
>>>> But all of this is still far in the future, AFAICT.
>>> 
>>> 
>>> I see two different things to consider.  One is the possibility of
>>> splitting the release into separate downloadable files.  That by
>>> itself seems simple enough, and we could do it if it seems useful.
>>> But as Axel says it doesn't seem all that useful.  It seems like it
>>> would make downloading a release more complex for 99% of people for
>>> the benefit of 1% of people.  That's not an impossible tradeoff but it
>>> needs to really be a big help for the 1%.  I don't see the benefit
>>> myself (but admittedly I have a fast Internet connection anyhow).
>>> 
>>> The more interesting thing to consider is that if we split the release
>>> like that, people will naturally start mixing and matching different
>>> versions, either on purpose or by accident.  That could be useful in
>>> some ways, but it is also much harder to support.  Considering how
>>> much trouble we have making a single unified release, it's not obvious
>>> to me that the Go team has the bandwidth to handle separate release
>>> cycles that need to work together.
>>> 
>>> Ian
>>> 
>>> 
>>> 
>>>>> On Mon, Jan 27, 2020 at 3:31 PM Volker Dobler 
>>>>>  wrote:
>>>>> 
>>>>> On Monday, 27 January 2020 12:27:35 UTC+1, changkun wrote:
>>>>>> 
>>>>>> Dear golang-nuts,
>>>>>> 
>>>>>> As https://github.com/golang/go/issues/27151, 
>>>>>> https://github.com/golang/go/issues/6853 and many relevant issues 
>>>>>> di

Re: [go-nuts] Re: Is it considered to separate runtime and cmd/compile irrelevant standard libraries out from the core language repo and hosted in a different module, eg. std?

[Robert Engels]

Doesn’t this piggyback on making the runtime a shared library so it can be 
updated without application recompile?

> On Jan 27, 2020, at 2:24 PM, Ian Lance Taylor  wrote:
> 
> On Mon, Jan 27, 2020 at 7:02 AM 'Axel Wagner' via golang-nuts
>  wrote:
>> 
>> The original message mentions size. The given list is 25MB/337MB 
>> uncompressed or 7MB/115MB compressed. So in terms of saved space, we are 
>> talking ~6%. It's not nothing, but it's also not a lot. Especially as you'll 
>> most likely need to download them right after anyway.
>> 
>> You also mentions splitting up the issue tracker. But note, that even though 
>> the various x-repos are already separate repositories, they still use the 
>> same issue-tracker. Clearly, having a unified issue tracker is perceived as 
>> a *benefit* to the Go team, not a detriment.
>> 
>> It might indeed be worth versioning the stdlib as modules at some point. I 
>> know that was discussed as part of the question of what "Go 2" means. Having 
>> the ability to bump major versions of the stdlib would enable to overhaul 
>> APIs at some point in the future. But note that the Go 1 stability promise 
>> stays in place, so again, neither in terms of size, nor in terms of 
>> splitting management, this would provide any benefits. Programs using Go v1 
>> stdlib packages will likely continue to exist into the far future, so even 
>> *if* there's at some point a v2 of the stdlib, v1 will still need to be 
>> shipped and supported (though likely the v1 API would wrap v2 packages, so 
>> the actual size increase would be moderate). But all of this is still far in 
>> the future, AFAICT.
> 
> 
> I see two different things to consider.  One is the possibility of
> splitting the release into separate downloadable files.  That by
> itself seems simple enough, and we could do it if it seems useful.
> But as Axel says it doesn't seem all that useful.  It seems like it
> would make downloading a release more complex for 99% of people for
> the benefit of 1% of people.  That's not an impossible tradeoff but it
> needs to really be a big help for the 1%.  I don't see the benefit
> myself (but admittedly I have a fast Internet connection anyhow).
> 
> The more interesting thing to consider is that if we split the release
> like that, people will naturally start mixing and matching different
> versions, either on purpose or by accident.  That could be useful in
> some ways, but it is also much harder to support.  Considering how
> much trouble we have making a single unified release, it's not obvious
> to me that the Go team has the bandwidth to handle separate release
> cycles that need to work together.
> 
> Ian
> 
> 
> 
>>> On Mon, Jan 27, 2020 at 3:31 PM Volker Dobler  
>>> wrote:
>>> 
>>> On Monday, 27 January 2020 12:27:35 UTC+1, changkun wrote:
 
 Dear golang-nuts,
 
 As https://github.com/golang/go/issues/27151, 
 https://github.com/golang/go/issues/6853 and many relevant issues 
 discussed, Go download is huge.
>>> 
>>> 
>>> Neither of these issues benefits from splitting the stdlib from the
>>> compiler and/or the "runtime" (whatever you mean by that).
>>> Separating the compiler from its stdlib seems strange at least
>>> and does not help, neither to increase development speed nor
>>> to increase convenience.
>>> 
>>> V.
>>> 
 
 
 The Go download contains everything in the main repo. Since Go modules are 
 now a success, is it considered to separate all runtime irrelevant 
 (meaning, no need runtime support to implement, e.g. net poller need 
 runtime support because there is a scheduler) libraries out of the main 
 repo?
 
 For instance, the following packages can be separated to be stored in a 
 different repository, called std module:
 
 std/archive
 std/bufio
 std/bytes
 std/compress
 std/container
 std/context
 std/crypto
 std/database
 std/encoding
 std/errors
 std/expvar
 std/flag
 std/go
 std/hash
 std/html
 std/image
 std/index
 std/io
 std/log
 std/math
 std/mime
 std/path
 std/plugin
 std/regexp
 std/sort
 std/strconv
 std/strings
 std/syscall
 std/testdata
 std/testing
 std/text
 std/unicode
 
 Say we have a separate repository called golang/std, import these packages 
 can have the same import path, instead of "std/io", go modules can 
 overwrite the import path exactly like what it did for 
 "github.com/user/mod/v2", set the imported std modules in go.mod file, 
 import as "io" directly.
 
 Thanks for reading.
 Changkun
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Robert Engels]

easy prototyping != scaling

Pick one. 



> On Jan 27, 2020, at 1:42 PM, Liam Breck  wrote:
> 
> 
> Go mindshare suffers if it's seen as worse for prototyping, and necessary 
> only for scale. Especially as there are more efficient (albeit more complex) 
> ways to achieve scale.
> 
>> On Mon, Jan 27, 2020, 10:55 AM Eric S. Raymond  wrote:
>> Michael Jones :
>> > Python, to its credit, has the nice inclusive property of extensible
>> > interpreters of being friendly to "hang things" onto just like ornaments on
>> > trees. By linking with C/C++-libraries and adding glue/shim code, anything
>> > can be used from Python. This facility and interpretive execution (as Eric
>> > Johnson observed above) makes it a natural for discovery and trying of
>> > things.
>> > 
>> > These virtues may seem less so in eventual deployments, at scale, and in
>> > other cases.
>> 
>> True, and connects directly to my recent experience report.
>> 
>> Python was the best language to prototype reposurgeon in,
>> but scale pressure forced the move to Go.
>> -- 
>> http://www.catb.org/~esr/";>Eric S. Raymond
>> 
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAKvHMgRSCF5MQW4CTew5kD4KKNkdt4X7c0X1gqDq6Bxyz98ssQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/BE504CA9-0E8F-4B09-8522-CDC7139F264B%40ix.netcom.com.

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Robert Engels]

Developer: not sure this is going to work... use Python for POC
SVP: Looks great! Ship it!
Developer: but wait...

> On Jan 27, 2020, at 11:05 AM, Michael Jones  wrote:
> 
> 
> Python, to its credit, has the nice inclusive property of extensible 
> interpreters of being friendly to "hang things" onto just like ornaments on 
> trees. By linking with C/C++-libraries and adding glue/shim code, anything 
> can be used from Python. This facility and interpretive execution (as Eric 
> Johnson observed above) makes it a natural for discovery and trying of 
> things. 
> 
> These virtues may seem less so in eventual deployments, at scale, and in 
> other cases.
> 
>> On Mon, Jan 27, 2020 at 8:48 AM 'Eric Johnson' via golang-nuts 
>>  wrote:
>> 
>>> On Sunday, January 26, 2020 at 10:27:35 PM UTC-8, pentel...@gmail.com wrote:
>>> 
>>> IMHO, golang didn't make a dent on key areas to become a language of choice 
>>> like big data (analytics, complex event processing, etc.) and consequently, 
>>> hot topics like artificial intelligence. Exactly areas where python excels 
>>> today.
>> 
>> At least when it comes to analytics, ML, AI, there are various languages at 
>> play in this space. Python isn't replacing "R" either. I don't think we 
>> should expect that it would either. One of Python's intended strengths for 
>> AI / ML / analytics involves REPL, which is specifically not a targeted 
>> feature of Go. On the other hand, when it comes to situations like the 
>> execution of AI / ML models in performance critical environments, I have 
>> heard / read of anecdotal evidence of teams opting for Go.
>> 
>> The question isn't whether Go making an impact for a specific area, it is 
>> whether it is making an impact where it is appropriate. For that, from what 
>> I see, the answer is overwhelmingly *yes*.
>> 
>> Eric.
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/1c8579d4-27b5-470b-a49d-1bca548bc8f7%40googlegroups.com.
> 
> 
> -- 
> Michael T. Jones
> michael.jo...@gmail.com
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CALoEmQzCEWPWhcnv6rt0e-BjhA3%3DRrswL%2B10CgAv0sL6cOgRbw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C19C18BC-3008-4135-B50C-4B55EBD84268%40ix.netcom.com.

Re: [go-nuts] Experience report on a large Python-to-Go translation

[robert engels]

Didn’t mean to imply anything negative - you accomplished what you set out to 
do. I “think in Java” and a lot of my initial problems with Go revolved around 
that, but even after doing some serious Go development - in the Go style - I 
can agree with many of your points.

I will admit my bias against anything written in Python… :)

> On Jan 26, 2020, at 10:09 PM, Eric Raymond  wrote:
> 
> 
> 
> On Sunday, January 26, 2020 at 9:06:47 PM UTC-5, Robert Engels wrote:
> I think trying to write Python in Go is problematic. 
> 
> Of course it is. If I were starting a Go program from scratch I certainly 
> wouldn't try to code as though the language were Python.
> 
> The real burden of my experience report is that *after I got past* the 
> initial and somewhat herky-jerky translation, some of the expressiveness 
> problems I tripped over *continued* to be expressiveness problems in 
> idiomatic Go.
> 
> Perhaps this would be more obvious if you had seen some of the gripes I had 
> in early drafts that I removed because I realized they were artifacts of 
> Pythonic thinking. 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d810807f-a1ab-4fb6-a303-439ec456c7b5%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/golang-nuts/d810807f-a1ab-4fb6-a303-439ec456c7b5%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/50D0DDD4-590E-4262-9DD2-8FEF8B3D7E43%40ix.netcom.com.

Re: [go-nuts] Re: Experience report on a large Python-to-Go translation

[Robert Engels]

I think trying to write Python in Go is problematic. You say you intentional 
did and didn’t worry about “improving” the code. Using interfaces and Go design 
patterns is not improving - it’s writing proper Go imo. 

> On Jan 26, 2020, at 6:14 PM, pboampo...@gmail.com wrote:
> 
> 
> > ===  Catchable exceptions require silly contortions ===
> 
> I think the community is aware of the problem but still trying to find a more 
> Go-like solution. Take a look at some of the proposals if you haven't:
> https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling.md
> https://github.com/golang/go/labels/error-handling
> 
> As you know, what we usually use now is a chain of:
> x, err = f(y); if err != nil { /* handle err */ }
> where *handle err* usually involves a return/goto/break, so that the nesting 
> level is independent of the number of steps.
> 
> But one can also use the panic/recover mechanism locally to shorten the above 
> into something like this:
> x, err = f(y); check(err)
> Example here: https://play.golang.org/p/Gli8_bgyuDS
> (Variations are possible, including error decoration and callback handlers.)
> 
> Sometimes panic/recover across different functions (like you did) is more 
> appropriate. The convention is not to use it across package boundaries.
> 
> > And all it would take to get there is two minor loosenings of restrictions.
> > 
> > 1. The panic function has a new property, "terminating". [...]
> 
> log.Fatal and os.Exit have the same problem. They are not "terminating 
> statements", so if you want them at the bottom of a function with result 
> parameters you have to add a panic("unreachable").
> But I think it's very rare not to have one of the "success" paths at the end; 
> in 8 years it happened to me like a couple of times. Do you really expect to 
> have throw() at the bottom of functions?
> 
> > 2. A recover() call is no longer required to be within the lexical frame of 
> > a defer().
> 
> Would it be less ugly like this? (with recover in the catch func.)
> 
> | defer catch("recoverable", func(err error) {
> | fmt.Println("Recover:", err)
> | })
> 
> > === Absence of iterators ===
> 
> Interesting proposal. The new expression accepted by "range" would be of type 
> "func() (T, bool)", called repeatedly to get the next element until false.
> While the available solution is verbose and uses 2 more variables, I don't 
> think its readability is so bad:
> 
> | for f := repo.commits();; {
> | commit, ok := f()
> | if !ok {
> | break
> | }
> | do_stuff_to(commit)
> | }
> 
> Of course if you need the index you have to add yet more cruft.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d5115a08-aadb-420e-912c-9145ea7ae611%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/E0B2E0BE-00CC-4447-A81E-DB2A9DE91911%40ix.netcom.com.

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Robert Engels]

Which is exactly what github.com/robaho/fixed and many others do!

> On Jan 26, 2020, at 10:34 AM, Michael Jones  wrote:
> 
> 
> ...thus the virtue of scaled integers. scaling by 100 makes cents whole, 
> scaling by 1*100 gives four decimal places beyond that. There is nothing 
> bad about floating point despite the reputation, it's just not the number 
> system from algebra; nor is binary floating point the same as decimal 
> floating point. The problems all start with false presumptions.
> 
>> On Sun, Jan 26, 2020 at 8:20 AM Robert Engels  wrote:
>> Just an FYI, often that is not correct. Many financial systems require 
>> fractional pennies due to the volume of transactions. Think about taxing 
>> stock exchanges the pennies add up quickly at any tax rate, so they use 
>> fractional pennies to reduce the size of the error bucket. 
>> 
>>>> On Jan 26, 2020, at 8:50 AM, Pat Farrell  wrote:
>>>> 
>>> 
>>> never use floating point if you are trying to represent money, say dollars 
>>> and cents or decimal values of the euro.
>>> Store the money as integer number of pennies.
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com.
> 
> 
> -- 
> Michael T. Jones
> michael.jo...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20AC2C62-3898-420C-805E-5A435CBF4391%40ix.netcom.com.

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Robert Engels]

Just an FYI, often that is not correct. Many financial systems require 
fractional pennies due to the volume of transactions. Think about taxing stock 
exchanges the pennies add up quickly at any tax rate, so they use 
fractional pennies to reduce the size of the error bucket. 

> On Jan 26, 2020, at 8:50 AM, Pat Farrell  wrote:
> 
> 
> never use floating point if you are trying to represent money, say dollars 
> and cents or decimal values of the euro.
> Store the money as integer number of pennies.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com.

Re: [go-nuts] keep just 2 decimal places in a float64

[Robert Engels]

You can use github.com/robaho/fixed :)

> On Jan 25, 2020, at 11:40 PM, Jamil Djadala  wrote:
> 
> On Sat, 25 Jan 2020 19:14:15 -0800 (PST)
> "Jason E. Aten"  wrote:
> 
>> 
>> https://play.golang.org/p/87bDubJxjHO
>> 
>> I'd like to truncate a float64 to just 2 decimal places (in base 10),
>> but math.Trunc is not helping me here... ideally I put 0.29 in and I
>> get 0.29 out.
>> Suggestions?  Playground examples appreciated.
> 
> you can use int types:
> https://play.golang.org/p/qLnynemd3S1
> -- 
> Jamil Djadala
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/20200126074011.0f17b4fa%40wolf.home.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4FB8D934-4C40-49D1-B748-4F2051C04B8C%40ix.netcom.com.

Re: [go-nuts] Re: Experience report on a large Python-to-Go translation

[Robert Engels]

Very in-depth and interesting. 

Although I agree with most of the points, I think a better use of interfaces 
would address some of your concerns, and have more maintainable code. 

Whenever I see a type switch it screams to me “use an interface and 
restructure. “

>> On Jan 25, 2020, at 4:43 AM, Brian Candler  wrote:
> 
> Very insightful.
> 
> I am relatively new to go, but I would like to make a few observations.
> 
> 1. When the issue of keyword arguments has come up before, usually someone 
> suggests passing a struct as the function argument.  Did you try this?  It 
> might be worth mentioning in your analysis, if only to give an example of why 
> it wasn't a good match.
> 
> 2. I notice some potential overlap between a couple of features you mention.  
> The first is map(f,x) for mapping a slice of arbitrary type (incidentally it 
> couldn't be called "map" for obvious reasons)  The second is using generator 
> functions as ranges.
> 
> It occurs to me that if you could initialize a slice from a generator 
> function, you'd have another way to achieve your map. The generator function 
> could also do filtering, making it more like a comprehension.
> 
> 3. A generator function would have to maintain its own thread of execution 
> independent from the recipient - much like a goroutine.  So this might end up 
> looking very similar to a function which stuffs values down a channel, which 
> is option 3 in the linked article.
> 
> The problem with garbage collection needs to be dealt with, which at the 
> simplest could be that the receiver closes the channel when done. Today this 
> would cause the sender to panic, so that would need to be dealt with - 
> perhaps some sort of "soft close".
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/fdd4882f-0c17-42f2-b301-ab65e5ce08a7%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C1720736-D9CC-4D98-9D54-5EF69CF1B9CD%40ix.netcom.com.

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

[Robert Engels]

See https://github.com/golang/go/issues/9849

Go has no limit, you use ulimit to control.



-Original Message-
>From: Kevin Chadwick 
>Sent: Jan 23, 2020 10:26 AM
>To: golang-nuts@googlegroups.com
>Subject: Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?
>
>On 2020-01-23 14:18, robert engels wrote:
>> There is nothing “special” about it - generally the Go process calls 
>> “malloc()” and fails with OOM (unable to expand the process memory size), 
>> but the OS can kill processes in a low system memory condition without them 
>> calling malloc (OOM killer kills the hogs). If you process is dying due to 
>> the OOM killer you have configuration problems.
>> 
>
>Because GO has it's own limit right? Whereas firefox/chrome didn't bother,
>likely because who are firefox/chrome to limit your systems memory use when it
>is the OS job. I guess GO limits it for panics or because Linux has a
>questionable default of unlimited or maybe because it supports so many
>platforms, otherwise it does not make any sense to me. The point is that if
>polling has race issues then I would investigate whether an OOM return/panic
>might be a timely indicator to the application/routine to clean house.
>
>> When malloc() fails in a GC system, it could be because the free space is 
>> fragmented. In a compacting and moving GC, it will shift objects around to 
>> make room (Go does not does this, most Java collectors do). Additionally, 
>> what I was primarily pointing out, rather than failing the GC will free 
>> “soft refs” to make room.
>
>Right, I believe tiny go does some compacting but you are better off without
>dynamic memory at all on a micro without an MMU. This compacting doesn't really
>solve the issue in a dynamic world with an MMU either, but just makes the 
>system
>more predictable, doesn't it?
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to golang-nuts+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/golang-nuts/7fea4901-16df-93e7-8a7c-9b95beb9a33e%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/58993198.1164.1579797754762%40wamui-bella.atl.sa.earthlink.net.

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

[robert engels]

There is nothing “special” about it - generally the Go process calls “malloc()” 
and fails with OOM (unable to expand the process memory size), but the OS can 
kill processes in a low system memory condition without them calling malloc 
(OOM killer kills the hogs). If you process is dying due to the OOM killer you 
have configuration problems.

When malloc() fails in a GC system, it could be because the free space is 
fragmented. In a compacting and moving GC, it will shift objects around to make 
room (Go does not does this, most Java collectors do). Additionally, what I was 
primarily pointing out, rather than failing the GC will free “soft refs” to 
make room.



> On Jan 23, 2020, at 6:13 AM, Kevin Chadwick  wrote:
> 
> On 2020-01-20 18:57, Robert Engels wrote:
>> This is solved pretty easily in Java using soft references and a hard memory 
>> cap. 
>> 
>> Similar techniques may work here.
> 
> One of the only things I dislike about GO compared to C is the arbitrary 
> memory
> allocation but it has great benefits in coding time and I expect you can 
> handle
> an array allocation panic etc. and keep track of your buffers.
> 
> I know that OpenBSD sets limits by default which need to be raised for
> chrome/firefox to prevent OOM death etc. which isn't the case on Linux without
> default limits (last I heard). I believe Linux kills the hogging process 
> instead!
> 
> I seem to remember OpenBSD devs saying the OS provides opportunity for Firefox
> to manage it's own OOM condition with these limits in place. I took that to 
> mean
> that Linux defaults have made it difficult to handle this properly in general,
> but I may lack understanding of the general issue?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/97c30a53-1b99-6424-fd00-9a7ae10a77f6%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8F45240D-E57E-4655-98BE-4E103A222405%40ix.netcom.com.

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

[robert engels]

This is actually SoftRef - which are like a WeakRef but they are only collected 
under memory pressure.

If the WeakRef package works, I assume it could be modified to enable “soft 
ref” like functionality. It was my understanding that you need GC/runtime 
support to truly make this work, but maybe they have an Unsafe/CGO way. I 
haven’t really researched the WeakRef packages for Go.



> On Jan 20, 2020, at 4:58 PM, Eric S. Raymond  wrote:
> 
> Robert Engels :
>> This is solved pretty easily in Java using soft references and a hard memory 
>> cap. 
> 
> That'd be nice, but the onnly weak-references package I've found doesn't seem 
> to allow
> more than one weakref per target. That's really annoying, because my use case 
> is
> a target object for a many-to-one mapping that should become GCable when the
> last of its source objects is GCed.
> 
> Is there a weakrefs implementation out tere that will do that?
> -- 
>   http://www.catb.org/~esr/";>Eric S. Raymond
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/20200120225853.GA75815%40thyrsus.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/A97065E1-66FD-42A5-9E6B-7AFA5CD96A68%40ix.netcom.com.

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

[Robert Engels]

This is solved pretty easily in Java using soft references and a hard memory 
cap. 

Similar techniques may work here. 

> On Jan 20, 2020, at 11:22 AM, Christian Mauduit  wrote:
> 
> Hi,
> 
> That is a generic question and I think that if you want to keep an approach 
> with a "global indicator of how much memory is used", your approach is OK. 
> You might also want to store this information of "should I throttle" in a 
> cache or something, the cache could be just a shared atomic flag.
> 
> But, from my experience, ugly OOMs can come very fast. So a ticker might just 
> not catch them. On modern hardware, you can allocate gigabytes of memory 
> within seconds, and hit the roof before your ticker based watcher notices 
> anything.
> 
> The best rules I have seen to enforce proper memory control is:
> 
> - channels are blessed as they are limited in size
> - bind everything else to a limit. More specifically, enfore limits for:
>  - maps (limit the number of entries)
>  - arrays
>  - generally speaking, anything which can *grow*
> 
> It is tedious, but on top of avoiding the "my ticker does not run often 
> enough" problem, you will also be able to take a decision (should I allocate 
> or not) based on local, actionable facts. The problem with the global 
> approach is that at some point, when the program is running out of memory, 
> you will not know *WHAT* is taking so much space. So you might end up 
> stopping allocating Foos when the problem is that there are too many Bars 
> allocated. I have seen this happening over and over.
> 
> So from my experience, just:
> 
> - observe, monitor your program to see what is taking most memory
> - enforce limitation on that part to ensure it is bounded
> - make this configurable as you will need to fine-tune it
> - write a test that would OOM without memory control
> 
> Sometimes the naive approach of those low-level limits does not work.
> 
> Example: you have a map[int64]string and you can affort ~1Gb for it, but the 
> size of string can range from 1 byte to 1Mb, with an average of 1Kb, and you 
> need to store 1M keys. With such a case, if I want to be serious about OOMs 
> and if it is a real problem, I would bite the bullet and simply track the 
> total amount of bytes stored. Maybe not the precise amount of data, summing 
> the length of all strings could be enough.
> 
> But the bottom line is -> I fear there is no "one size fits all" solution for 
> this.
> 
> It is expected that library providers do not implement this as they wish to 
> offer generic multi-purpose tools, and memory enforcement is rather an 
> application, final-product requirement.
> 
> Best luck, OOMs are hard.
> 
> Christian.
> 
>> On 20/01/2020 09:22, Urjit Singh Bhatia wrote:
>> Hi folks,
>> I am trying to figure out if someone has a decent solution for max memory 
>> usage/mem-pressure so far. I went through some of the github issues related 
>> to this (SetMaxHeap proposals and related discussions) but most of them are 
>> still under review:
>>  * https://go-review.googlesource.com/c/go/+/46751
>>  * https://github.com/golang/go/issues/16843
>>  * https://github.com/golang/go/issues/23044
>> My use case is that I have an in-memory data store and producers can 
>> potentially drive it to OOM, causing the server to drop all the data. I'd 
>> like to avoid it as much as possible by potentially informing the producers 
>> to slow down/refuse new jobs for a bit till some data is drained if I has 
>> some sort of a warning system. I think rabbitmq does something similar  
>> https://www.rabbitmq.com/memory.html
>>  * How are others in the community handling these situations? It seems
>>like most of database-ish implementations will just OOM and die.
>>  * Has anyone implemented a mem pressure warning mechanism?
>> Does something like this make sense for detecting high mem usage? (Of course 
>> other programs can randomly ask for memory from the OS so this isn't going 
>> to be leak-proof)
>>funcmemWatcher(highWatermarkuint64, memCheckIntervaltime.Duration) {
>>s:=&runtime.MemStats{}
>>// Count how many times we are consecutively beyond a mem watermark
>>highWatermarkCounter:=-1
>>forceGCInterval:=10
>>forrangetime.NewTicker(memCheckInterval).C {
>>runtime.ReadMemStats(s)
>>ifs.NextGC >=highWatermark {
>>log.Println("Approaching highWatermark") // Approaching highWatermark
>>continue
>>}
>>// Crossing highWatermark
>>ifs.HeapAlloc >=highWatermark {
>>ifhighWatermarkCounter <0{
>>// Transitioning beyond highWatermark
>>log.Println("High mem usage detected! Crossed high watermark")
>>// Start counters
>>highWatermarkCounter=0
>>forceGCInterval=10
>>log.Println("Reject/Throttle new work till usage reduces")
>>continue
>>} else{
>>// Still above highWatermark
>>highWatermarkCounter++
>>ifhighWatermarkCounter >=forceGCInterval {
>>log.Println("Forcing GC...")
>>runtime.GC()
>>fo

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Robert Engels]

Porsche’s sales are a tiny fraction of auto sales, but most in the industry 
consider it to be the best platform - and only wish they could emulate / afford 
to be in that conversation. 

> On Jan 16, 2020, at 2:10 PM, Liam  wrote:
> 
> 
> Open source is a rapidly growing movement across virtually all languages, but 
> still a small minority of all software. So the Trends graph is more revealing 
> than Github activity. Assembly language use on Github also increased by ~150%
> 
> There's no question that Go use is growing rather quickly. It's the mindshare 
> that I'm preoccupied with, because I'd expect that to impact Alphabet Inc's 
> budgeting calculus.
> 
> On a golang-dev thread recently, I raised the point that talking about "Go 2" 
> may have created a widespread misperception (esp among non-users) that a 
> backwards incompatible "v2.0" language is planned.
> 
> This comment on the last Redmonk survey is telling about Go's public 
> perception:
> 
> Go (-1): For the second run in a row, Go dropped one spot, this time out of a 
> tie with R for 15th back to 16th on our list. To be sure, placement in the 
> top twenty is by itself a remarkable achievement; many popular, widely used 
> and beloved languages lay well behind it. But for all of its credibility and 
> usage in widely used, popular projects, Go’s lack of versatility – perceived 
> or otherwise – has limited its upside. Go has remained a solidly top twenty 
> language, but has never placed higher than 14th, and that for only a single 
> quarter. It will also be interesting to see if any of the controversy 
> surrounding Go’s future direction – and the community’s input or lack thereof 
> into that – has any observable impact on the language’s traction moving 
> forward.
> 
> From https://redmonk.com/sogrady/2019/07/18/language-rankings-6-19/
> 
>> On Thursday, January 16, 2020 at 7:33:36 AM UTC-8, Amnon Baron Cohen wrote:
>> Go was originally conceived as a niche language. And if it does what we 
>> need, then I don't think 
>> we need to be particularly bothered if other languages are more "popular".
>> 
>> But when looking at language popularity, I am not sure that the number of 
>> google searches is the most meaningful metric.
>> 
>> Lines of code on github could be more interesting.
>> 
>> FWIW: Githubs octoverse shows shows a 147% growth in Go usage last year.
>> 
>> And more interesting growth stats can be found on the Go blog 
>> https://blog.golang.org/8years
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/5f40c9c1-2ca3-419e-b410-f0f835d63d04%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0920AC1A-AFAC-482E-BFFD-45CE36ACAE11%40ix.netcom.com.

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Robert Engels]

Well said. 

> On Jan 16, 2020, at 9:58 AM, Michael Jones  wrote:
> 
> 
> How global mindshare develops is something that I know quite a bit about 
> through leadership and engineering experience in multiple billion user 
> projects. 
> 
> One key lesson for me was that you reach a point where the audience you 
> originally wanted to serve (or refocused to serve) has been served. That’s 
> when the debate of “more for this group” or “something for other groups” 
> starts with vigor. 
> 
> There is a natural desire to grow but my advice here, after looking back 
> honestly, is that the way to move forward is to be so excellent at some 
> aspect(s) that users become effective missionaries. This is the only scaling 
> mechanism at scale (other than force in unusually controlled scenarios).
> 
> Looking forward (i.e. guessing) maybe Go needs new greatness in what it 
> already is and has by way of an “encyclopedia” of well-loved solutions. 
> Imagine a guide to 100 top uses with links to tools, samples for each, and 
> lots of details so that anyone wanting to build a static or dynamic web 
> server, ftp server, ssh client, mail processor, ... would have complete 
> guides from start to finish. 
> 
> Maybe existing solutions are sufficient or maybe they could be better. If 
> they have room for improvement then my guess is that this kind of beginner 
> hand holding might be the most effective investment for user growth. 
> 
> Just a guess,
> Michael
> 
>> On Thu, Jan 16, 2020 at 7:33 AM Amnon Baron Cohen  wrote:
>> Go was originally conceived as a niche language. And if it does what we 
>> need, then I don't think 
>> we need to be particularly bothered if other languages are more "popular".
>> 
>> But when looking at language popularity, I am not sure that the number of 
>> google searches is the most meaningful metric.
>> 
>> Lines of code on github could be more interesting.
>> 
>> FWIW: Githubs octoverse shows shows a 147% growth in Go usage last year.
>> 
>> And more interesting growth stats can be found on the Go blog 
>> https://blog.golang.org/8years
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/28b8066f-876b-41f6-b249-94dc4f255347%40googlegroups.com.
> -- 
> Michael T. Jones
> michael.jo...@gmail.com
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CALoEmQy-c1eH-bCH1vmTyMdf3_NMt31v6cWSdD8abha1OT%2BF%3Dw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/07F3F0E3-3E3F-49CF-BFA2-C63E68AA9BFE%40ix.netcom.com.

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Robert Engels]

Please look deeper into how these "trends" are calculated.For example, if everyone that uses product Y can't figure out how feature X works, and they search for it. Product Y will be showing growth...-Original Message-
From: Liam 
Sent: Jan 15, 2020 4:18 PM
To: golang-nuts 
Subject: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

My point is that Go should be rising, since Java & C# are falling. Python has gained significant mindshare, and Go is way better.I think something's amiss with the public perception of Go, but it's hard to say what.On Wednesday, January 15, 2020 at 12:34:54 PM UTC-8, Liam wrote:Google Trends graph showing past 5y of Java, Python, C#, Node.js, Gohttps://trends.google.com/trends/explore?date=today%205-y&q=%2Fm%2F07sbkfb,%2Fm%2F05z1_,%2Fm%2F07657k,%2Fm%2F0bbxf89,%2Fm%2F09gbxjr



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8dc0b035-d5fe-4ccf-87d3-84c0978c881d%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/286205293.3738.1579129075142%40wamui-lucy.atl.sa.earthlink.net.

Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

[Robert Engels]

That is why this should be tightened down imo. It’s obscure cruft that needs to 
be accounted for when refactoring. 

> On Jan 15, 2020, at 2:23 PM, Tom Payne  wrote:
> 
> 
> Thanks all for the insight and explanation. Could I suggest tweaking the Go 
> language specification to emphasize the separation, so it reads:
> 
>"when evaluating the operands of an expression, assignment, or return 
> statement, then all function calls, method calls, and communication 
> operations are evaluated in lexical left-to-right order. Any operands that 
> are variables take the value of the variable after all function calls, 
> methods calls, and communication operations have been evaluated."
> 
> The presence or absence of function calls does make a difference here. In 
> this example the use of the identity() function changes the result of f():
> 
>   func identity(i int) int {
>   return i
>   }
> 
>   func modify(i *int) error {
>   *i = 1
>   return nil
>   }
> 
>   func f() (int, error) {
>   i := 0
>   return identity(i), modify(&i) // returns 0, nil
>   }
> 
> 
>> On Wed, 15 Jan 2020 at 20:50, Robert Engels  wrote:
>> I disagree here. Knowing the order of operations/evaluation is part of 
>> knowing the language. If that’s the case it should be undefined (which 
>> forces a rewrite) not chose a behavior that seems contrary to the 
>> documentation and common practice in this area. 
>> 
>>>> On Jan 15, 2020, at 1:39 PM, Axel Wagner  
>>>> wrote:
>>>> 
>>> 
>>> I don't think "clearly this behavior is not what you would want" is a valid 
>>> generalization.
>>> 
>>>   type MyStruct{
>>>   // …
>>>   }
>>> 
>>>   func (x *MyStruct) initialize() error {
>>>   // does some common verification and populates some internal data 
>>> structures
>>>   return nil
>>>   }
>>> 
>>>   func New() (*MyStruct, error) {
>>>   x := &MyStruct{
>>>// default fields
>>>   }
>>>   return x, x.initialize()
>>>   }
>>> 
>>>   func NewWithFoo(f Foo) (*MyStruct, error) {
>>>    x := &MyStruct{
>>> Foo: f,
>>>}
>>>return x, x.intialize()
>>>   }
>>> 
>>> You can say that you shouldn't do something like this, but it's IMO 
>>> reasonable code to write and it expects different behavior.
>>> 
>>> However, as per my previous e-mail, I would still prefer to rewrite this, 
>>> of course :)
>>> 
>>> 
>>>> On Wed, Jan 15, 2020 at 8:33 PM Robert Engels  
>>>> wrote:
>>>> I think the way to look at it is “what would be the behavior of if you 
>>>> were inline creating and initializing a struct containing both values” - 
>>>> clearly this behavior is not what you would want or expect. 
>>>> 
>>>> > On Jan 15, 2020, at 1:25 PM, Paul Jolly  wrote:
>>>> > 
>>>> > 
>>>> >> 
>>>> >>  "when evaluating the operands of an expression, assignment, or return 
>>>> >> statement, all function calls, method calls, and communication 
>>>> >> operations are evaluated in lexical left-to-right order."
>>>> > 
>>>> > My understanding goes as follows: the operands of the return statement
>>>> > are i and modify(&i). The comma after "return statement" in the above
>>>> > sentence is then important: because the only "function calls, method
>>>> > calls, and communication operations" in that list of operands are (is)
>>>> > modify(&i).
>>>> > 
>>>> > Hence when i (as in the first operand) is evaluated is not specified.
>>>> > And therefore it's dangerous to rely on it being one value or another
>>>> > (which it could be given the example you provide).
>>>> > 
>>>> > There was even some discussion at the London Gophers November 2019
>>>> > meetup (where the conundrum was very related
>>>> > https://docs.google.com/presentation/d/e/2PACX-1vQ07uP_ldnSYzpaNb5AlZZ-_tL2mZuoNfQgxvsTKSM4aglYR-nuvyrZ8nK__r3YQTo1vqH-Hmax3aXs/pub?slide=id.g6239d05d0e_0_1)
>>>> > about whether it would be possible to statically flag potential errors
>>>> > like this.
>>>> > 
>>>> > -- 
>>>>

Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

[Robert Engels]

I disagree here. Knowing the order of operations/evaluation is part of knowing 
the language. If that’s the case it should be undefined (which forces a 
rewrite) not chose a behavior that seems contrary to the documentation and 
common practice in this area. 

> On Jan 15, 2020, at 1:39 PM, Axel Wagner  
> wrote:
> 
> 
> I don't think "clearly this behavior is not what you would want" is a valid 
> generalization.
> 
>   type MyStruct{
>   // …
>   }
> 
>   func (x *MyStruct) initialize() error {
>   // does some common verification and populates some internal data 
> structures
>   return nil
>   }
> 
>   func New() (*MyStruct, error) {
>   x := &MyStruct{
>// default fields
>   }
>   return x, x.initialize()
>   }
> 
>   func NewWithFoo(f Foo) (*MyStruct, error) {
>x := &MyStruct{
> Foo: f,
>}
>return x, x.intialize()
>   }
> 
> You can say that you shouldn't do something like this, but it's IMO 
> reasonable code to write and it expects different behavior.
> 
> However, as per my previous e-mail, I would still prefer to rewrite this, of 
> course :)
> 
> 
>> On Wed, Jan 15, 2020 at 8:33 PM Robert Engels  wrote:
>> I think the way to look at it is “what would be the behavior of if you were 
>> inline creating and initializing a struct containing both values” - clearly 
>> this behavior is not what you would want or expect. 
>> 
>> > On Jan 15, 2020, at 1:25 PM, Paul Jolly  wrote:
>> > 
>> > 
>> >> 
>> >>  "when evaluating the operands of an expression, assignment, or return 
>> >> statement, all function calls, method calls, and communication operations 
>> >> are evaluated in lexical left-to-right order."
>> > 
>> > My understanding goes as follows: the operands of the return statement
>> > are i and modify(&i). The comma after "return statement" in the above
>> > sentence is then important: because the only "function calls, method
>> > calls, and communication operations" in that list of operands are (is)
>> > modify(&i).
>> > 
>> > Hence when i (as in the first operand) is evaluated is not specified.
>> > And therefore it's dangerous to rely on it being one value or another
>> > (which it could be given the example you provide).
>> > 
>> > There was even some discussion at the London Gophers November 2019
>> > meetup (where the conundrum was very related
>> > https://docs.google.com/presentation/d/e/2PACX-1vQ07uP_ldnSYzpaNb5AlZZ-_tL2mZuoNfQgxvsTKSM4aglYR-nuvyrZ8nK__r3YQTo1vqH-Hmax3aXs/pub?slide=id.g6239d05d0e_0_1)
>> > about whether it would be possible to statically flag potential errors
>> > like this.
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google Groups 
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to golang-nuts+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/golang-nuts/CACoUkn7KA0Z-TDypdvM%3Du%3DyVPHuHFmtD%3DiTV2c98Vm%3Dqn4NcPw%40mail.gmail.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/9EC93CAE-B11C-45E3-9D01-AC4AE5769BEE%40ix.netcom.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1388A9A2-F321-459A-893E-6D90D42E7255%40ix.netcom.com.

Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

[Robert Engels]

I think the way to look at it is “what would be the behavior of if you were 
inline creating and initializing a struct containing both values” - clearly 
this behavior is not what you would want or expect. 

> On Jan 15, 2020, at 1:25 PM, Paul Jolly  wrote:
> 
> 
>> 
>>  "when evaluating the operands of an expression, assignment, or return 
>> statement, all function calls, method calls, and communication operations 
>> are evaluated in lexical left-to-right order."
> 
> My understanding goes as follows: the operands of the return statement
> are i and modify(&i). The comma after "return statement" in the above
> sentence is then important: because the only "function calls, method
> calls, and communication operations" in that list of operands are (is)
> modify(&i).
> 
> Hence when i (as in the first operand) is evaluated is not specified.
> And therefore it's dangerous to rely on it being one value or another
> (which it could be given the example you provide).
> 
> There was even some discussion at the London Gophers November 2019
> meetup (where the conundrum was very related
> https://docs.google.com/presentation/d/e/2PACX-1vQ07uP_ldnSYzpaNb5AlZZ-_tL2mZuoNfQgxvsTKSM4aglYR-nuvyrZ8nK__r3YQTo1vqH-Hmax3aXs/pub?slide=id.g6239d05d0e_0_1)
> about whether it would be possible to statically flag potential errors
> like this.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CACoUkn7KA0Z-TDypdvM%3Du%3DyVPHuHFmtD%3DiTV2c98Vm%3Dqn4NcPw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9EC93CAE-B11C-45E3-9D01-AC4AE5769BEE%40ix.netcom.com.

Re: [go-nuts] [ANN] github.com/perillo/workon command

[Robert Engels]

You might find visual studio code appealing for your Go development. 

> On Jan 15, 2020, at 11:58 AM, Manlio Perillo  wrote:
> 
> 
> Hi.
> 
> I have several relatively small personal projects.  Every time I want to work 
> on them I need to launch a new terminal emulator and then open all the source 
> files with an editor.
> Doing it N times starts to become annoying.
> 
> For this reason I wrote the workon command.
> It will automatically start a terminal emulator at a Go specified project 
> path and open all [1] the project files with an editor.
> 
> It is available at https://github.com/perillo/workon.
> 
> Feedback is welcomed.
> Currently there are some issues, described in the README.md and main.go file.
> 
> [1] Use the equivalent of `go list ./...`
> 
> 
> Thanks
> Manlio Perillo
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/21921938-1a02-4bb3-a42e-5127d9617717%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/350B974B-525D-4D4E-9F0F-33FE6396B362%40ix.netcom.com.

[go-nuts] performance improvements

[robert engels]

Shout out to the Go team on a job well done. I took the time to re-run some 
benchmarks, here are the results (both 1.9.7 and 1.13.5 re-run on same hardware 
& OS).

>From fixed 

iMac:tmp robertengels$ benchcmp bench_1_9_7.txt bench_1_13_15.txt 
benchmark old ns/op new ns/op delta
BenchmarkAddFixed-8   1.29  1.01  -21.71%
BenchmarkAddDecimal-8 352   249   -29.26%
BenchmarkAddBigInt-8  15.3  14.8  -3.27%
BenchmarkAddBigFloat-880.6  80.1  -0.62%
BenchmarkMulFixed-8   4.40  4.60  +4.55%
BenchmarkMulDecimal-8 71.8  73.8  +2.79%
BenchmarkMulBigInt-8  17.8  17.4  -2.25%
BenchmarkMulBigFloat-837.1  35.4  -4.58%
BenchmarkDivFixed-8   4.75  4.63  -2.53%
BenchmarkDivDecimal-8 882   769   -12.81%
BenchmarkDivBigInt-8  56.8  45.4  -20.07%
BenchmarkDivBigFloat-8121   109   -9.92%
BenchmarkCmpFixed-8   1.02  0.50  -50.69%
BenchmarkCmpDecimal-8 8.42  8.03  -4.63%
BenchmarkCmpBigInt-8  5.93  5.44  -8.26%
BenchmarkCmpBigFloat-85.11  5.63  +10.18%
BenchmarkStringFixed-860.1  58.6  -2.50%
BenchmarkStringNFixed-8   59.0  57.2  -3.05%
BenchmarkStringDecimal-8  259   214   -17.37%
BenchmarkStringBigInt-8   164   126   -23.17%
BenchmarkStringBigFloat-8 465   433   -6.88%
BenchmarkWriteTo-845.1  38.4  -14.86%





-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5267D756-D269-4ABC-B8EB-D7C98D29F82E%40ix.netcom.com.

Re: [go-nuts] How to reduce the fluctuation of micro benchmark results

[robert engels]

Take a look at https://github.com/golang/go/issues/24735 


You probably want to increase benchtime to a larger value, even 10x.

> On Jan 7, 2020, at 8:44 PM, Xiangdong JI  wrote:
> 
> BTW. 'drop cache' is made, while setting cpu governor is not supported by the 
> platform. 
> Thanks.
> 
> On Wednesday, January 8, 2020 at 10:19:25 AM UTC+8, Xiangdong JI wrote:
> Hi,
> 
> Significant fluctuation is observed when running micro benchmarks, even with 
> the same version of go, e.g the following numbers from math/big package, 
> could anyone please share practices that may help generate stable result?
> 
> 
> FloatAdd/1-64 669ns ± 0% 676ns ± 1%+1.05%  
> (p=0.008 n=5+5)
> FloatAdd/10-64   5.29µs ± 1%5.56µs ± 0%+5.08%  
> (p=0.008 n=5+5)
> FloatSub/10-64125ns ± 0% 125ns ± 0%  ~ 
> (p=0.444 n=5+5)
> FloatSub/100-64   126ns ± 0% 126ns ± 0%  ~ 
> (all equal)
> FloatSub/1000-64  150ns ± 0% 150ns ± 0%  ~ 
> (all equal)
> FloatSub/1-64 393ns ± 0% 447ns ± 0%   +13.79%  
> (p=0.008 n=5+5)
> FloatSub/10-64   3.20µs ± 0%3.20µs ± 0%  ~ 
> (p=0.056 n=5+5)
> ParseFloatSmallExp-6425.3µs ±15%23.8µs ± 0%-6.26%  
> (p=0.016 n=5+5)
> ParseFloatLargeExp-6484.7µs ± 0%84.8µs ± 1%  ~ 
> (p=0.690 n=5+5)
> GCD10x10/WithoutXY-64 166ns ± 0% 165ns ± 0%-0.60%  
> (p=0.029 n=4+4)
> GCD10x10/WithXY-64746ns ± 1% 747ns ± 1%  ~ 
> (p=0.810 n=4+5)
> GCD10x100/WithoutXY-64438ns ± 0% 439ns ± 0%  ~ 
> (p=0.730 n=5+5)
> GCD10x100/WithXY-64  2.35µs ±14%2.23µs ± 2%  ~ 
> (p=0.175 n=5+5)
> GCD10x1000/WithoutXY-64  1.16µs ±13%1.37µs ±43%  ~ 
> (p=0.310 n=5+5)
> GCD10x1000/WithXY-64 6.20µs ±59%4.71µs ±29%  ~ 
> (p=0.841 n=5+5)
> GCD10x1/WithoutXY-64 16.8µs ±52%11.7µs ±46%  ~ 
> (p=0.151 n=5+5)
> GCD10x1/WithXY-6458.6µs ±53%80.2µs ±53%  ~ 
> (p=0.310 n=5+5)
> GCD10x10/WithoutXY-6472.0µs ±20%77.1µs ±16%  ~ 
> (p=0.421 n=5+5)
> GCD10x10/WithXY-64169µs ±19% 271µs ±28%   +60.47%  
> (p=0.016 n=5+5)
> GCD100x100/WithoutXY-64  2.37µs ±18%2.31µs ± 8%  ~ 
> (p=0.841 n=5+5)
> GCD100x100/WithXY-64 4.22µs ± 7%4.05µs ± 1%  ~ 
> (p=0.556 n=5+4)
> GCD100x1000/WithoutXY-64 3.73µs ± 7%4.02µs ±27%  ~ 
> (p=0.841 n=5+5)
> GCD100x1000/WithXY-647.44µs ± 3%   10.67µs ±22%   +43.33%  
> (p=0.008 n=5+5)
> GCD100x1/WithoutXY-6432.4µs ±63%20.8µs ± 3%   -35.83%  
> (p=0.016 n=5+4)
> GCD100x1/WithXY-64   71.3µs ±93%   118.5µs ±44%  ~ 
> (p=0.151 n=5+5)
> GCD100x10/WithoutXY-64199µs ±56% 195µs ±29%  ~ 
> (p=1.000 n=5+5)
> GCD100x10/WithXY-64   459µs ±33% 892µs ±60%   +94.33%  
> (p=0.016 n=5+5)
> GCD1000x1000/WithoutXY-6422.8µs ±14%22.5µs ±10%  ~ 
> (p=1.000 n=5+5)
> GCD1000x1000/WithXY-64   39.8µs ± 0%43.9µs ±10%   +10.26%  
> (p=0.008 n=5+5)
> GCD1000x1/WithoutXY-64   41.6µs ± 0%56.0µs ±16%   +34.70%  
> (p=0.008 n=5+5)
> GCD1000x1/WithXY-64   154µs ± 0% 302µs ±29%   +96.15%  
> (p=0.008 n=5+5)
> GCD1000x10/WithoutXY-64   240µs ± 0% 268µs ±11%   +11.64%  
> (p=0.016 n=4+5)
> GCD1000x10/WithXY-64 2.51ms ±60%   9.28ms ±197%  +270.21%  
> (p=0.032 n=5+5)
> GCD1x1/WithoutXY-64   545µs ± 0% 549µs ± 0%+0.69%  
> (p=0.008 n=5+5)
> GCD1x1/WithXY-64 1.20ms ± 0%1.20ms ± 1%  ~ 
> (p=0.222 n=5+5)
> GCD1x10/WithoutXY-64 1.40ms ± 0%1.67ms ±27%   +19.66%  
> (p=0.008 n=5+5)
> GCD1x10/WithXY-6411.7ms ± 9%11.4ms ±14%  ~ 
> (p=0.841 n=5+5)
> GCD10x10/WithoutXY-6438.8ms ± 0%38.8ms ± 0%  ~ 
> (p=0.690 n=5+5)
> GCD10x10/WithXY-64   84.9ms ± 0%85.0ms ± 0%  ~ 
> (p=0.905 n=5+4)
> Hilbert-64   1.93ms ± 0%1.93ms ± 1%  ~ 
> (p=1.000 n=4+5)
> Binomial-64  3.42µs ± 2%3.59µs ± 8%  ~ 
> (p=0.095 n=5+5)
> QuoRem-644.55µs ± 0%4.55µs ± 0%  ~ 
> (p=0.659 n=5+5)
> Exp-64   17.1ms ± 0%17.1ms ± 0%  ~ 
> (p=0.841 n=5+5)
> Exp2-64  17.1ms ± 0%17.1ms ± 0%  ~ 
> (p=0.421 n=5+5)
> Bitset-6420.8ns ± 0%20.8ns ± 0%  ~ 
> (p=1.000 n=5+5)
> BitsetNeg-64  

Re: [go-nuts] Re: distributed runtime

[Robert Engels]

I think you want to use “nats streaming”. It does throttling via ACKs. 

> On Jan 6, 2020, at 7:35 AM, Jason E. Aten  wrote:
> 
> 
>>> On Mon, Jan 6, 2020 at 6:24 AM Robert Engels  wrote:
>>> I’m pretty sure Nats.io has all of the same delivery guarantees as 
>>> rabbitmq. What design is not accommodated by mats?
>>  
>> Any design that needs flow control is not accommodated by Nats.io. I've used 
>> Nats extensively and consider it fundamentally flawed for data conveyance. 
>> It is only suitable for very light weight control messages that do not need 
>> to be delivered; where the application logic handles the retries instead of 
>> TCP.  Push any kind of data through Nats, and its lack of flow control means 
>> it will drop most all of your data on the floor.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/FB78E62B-9310-4462-ADCA-59A184218DDB%40ix.netcom.com.

Re: [go-nuts] Re: distributed runtime

[Robert Engels]

I’m pretty sure Nats.io has all of the same delivery guarantees as rabbitmq. 
What design is not accommodated by mats?

> On Jan 6, 2020, at 4:56 AM, ffm2...@web.de wrote:
> 
> 
> 
>> 
>> However, if you want to apply supervision for distributed Go channels you 
>> have to change from channels to actors - at least behind the scenes of the 
>> distributed channels. AFAIK, no one has done something like that for 
>> anything writen in Go. Nats is very reliable, but does not provide 
>> guaranteed "all or nothing" delivery.
> 
> Coming to think of it you could make use of Rabbitmq as the distributed 
> channel provider. It is written in Erlang and fulfills this "all or nothing" 
> requirement. Bindings for Go already exist: 
> https://godoc.org/github.com/streadway/amqp
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/52a565de-6c95-4a28-b8fd-1fc5bb6b1fb6%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F4B07F85-D655-468A-974F-FCC3A6BD621B%40ix.netcom.com.

Re: [go-nuts] A question !

[robert engels]

Well, don’t benchmark it, bookmark it :)

> On Jan 4, 2020, at 5:51 PM, robert engels  wrote:
> 
> Please everyone just benchmark this page 
> https://www.techempower.com/benchmarks/ 
> <https://www.techempower.com/benchmarks/>
> 
> Rant ON... Python is not suitable for any high volume server application. It 
> is designed for simple scripting. Rant OFF.
> 
>> On Jan 4, 2020, at 5:41 PM, Justin Israel > <mailto:justinisr...@gmail.com>> wrote:
>> 
>> 
>> 
>> On Sunday, January 5, 2020 at 7:28:24 AM UTC+13, Motaz Hejaze wrote:
>> 
>> 1 - Is there really a big performance ( speed ) difference between using 
>> Python OR Golang in backend web development ? 
>> 
>> There are a number of reasons why Go could out-perform Python when comparing 
>> django to a Go web backend. 
>> Firstly, in order to scale django across multiple cores on the same host, 
>> one generally deploys a python web framework under uwsgi which handles 
>> spawning multiple instances of the application and load balancing th 
>> request. Python is effectively single threaded because of the GIL so this is 
>> the solution for production python web servers. 
>> Go on the other hand can use all of the cores with just a single process and 
>> can be deployed directly. 
>> 
>> Aside from that, there could be other areas where Go performs better, 
>> depending on what your application is doing, because it is compiled and uses 
>> static types as opposed to python being interpreted and constantly using 
>> reflection. One can mitigate some of these performance issues with compiled 
>> python extensions. But again, it depends. 
>>  
>> 4 - I know that go comes with builtin web server , is it reliable for 
>> production ? or shall we connect go with classic web servers aka ( nginx , 
>> apache ) ?
>> 
>> The stdlib web server is production grade and can be used to deploy the 
>> binary directly without the need for a proxy.
>> But the use of a proxy can still be valid for situations where you want load 
>> balancing, static file caching, or TLS terminatation, as a few examples. 
>>  
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com 
>> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/f9e386d9-007a-4b1e-8841-474e098fbfeb%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/f9e386d9-007a-4b1e-8841-474e098fbfeb%40googlegroups.com?utm_medium=email&utm_source=footer>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/9803C50D-AB5B-469A-825A-95AABE783BC8%40ix.netcom.com
>  
> <https://groups.google.com/d/msgid/golang-nuts/9803C50D-AB5B-469A-825A-95AABE783BC8%40ix.netcom.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/E0DE7892-56FD-4760-BC52-09298FE42A48%40ix.netcom.com.

Re: [go-nuts] A question !

[robert engels]

Please everyone just benchmark this page 
https://www.techempower.com/benchmarks/ 


Rant ON... Python is not suitable for any high volume server application. It is 
designed for simple scripting. Rant OFF.

> On Jan 4, 2020, at 5:41 PM, Justin Israel  wrote:
> 
> 
> 
> On Sunday, January 5, 2020 at 7:28:24 AM UTC+13, Motaz Hejaze wrote:
> 
> 1 - Is there really a big performance ( speed ) difference between using 
> Python OR Golang in backend web development ? 
> 
> There are a number of reasons why Go could out-perform Python when comparing 
> django to a Go web backend. 
> Firstly, in order to scale django across multiple cores on the same host, one 
> generally deploys a python web framework under uwsgi which handles spawning 
> multiple instances of the application and load balancing th request. Python 
> is effectively single threaded because of the GIL so this is the solution for 
> production python web servers. 
> Go on the other hand can use all of the cores with just a single process and 
> can be deployed directly. 
> 
> Aside from that, there could be other areas where Go performs better, 
> depending on what your application is doing, because it is compiled and uses 
> static types as opposed to python being interpreted and constantly using 
> reflection. One can mitigate some of these performance issues with compiled 
> python extensions. But again, it depends. 
>  
> 4 - I know that go comes with builtin web server , is it reliable for 
> production ? or shall we connect go with classic web servers aka ( nginx , 
> apache ) ?
> 
> The stdlib web server is production grade and can be used to deploy the 
> binary directly without the need for a proxy.
> But the use of a proxy can still be valid for situations where you want load 
> balancing, static file caching, or TLS terminatation, as a few examples. 
>  
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/f9e386d9-007a-4b1e-8841-474e098fbfeb%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9803C50D-AB5B-469A-825A-95AABE783BC8%40ix.netcom.com.

Re: [go-nuts] Locking goroutine to "main" thread

[robert engels]

Also, even simpler - just remove the time.Atfter case() and use a default: - 
but the problem is you will spin a single thread at 100% cpu - you really need 
to use a blocking sdl.WaitEvent()

> On Jan 3, 2020, at 6:04 PM, robert engels  wrote:
> 
> You only need a single thread locked to the UI thread.
> 
> Use one Go routine locked to the UI thread. Put events onto a channel.
> 
> Have another Go routine read from the channel, and the app command channel 
> using select.
> 
> 
> 
>> On Jan 3, 2020, at 5:28 PM, bucha...@gmail.com <mailto:bucha...@gmail.com> 
>> wrote:
>> 
>> Whether the UI thread is on the main thread or a separate thread, a single 
>> UI thread needs to process events from two sources: the OS window and from 
>> my application. Having one loop process both sources (without support from 
>> the Go runtime scheduler) is the part I'm struggling with.
>> 
>> My latest iteration is something like:
>> 
>> for {
>>   // PollEvent doesn't block/sleep
>> 
>>   for ev := sdl.PollEvent(); ev != nil; ev = sdl.PollEvent {
>> // handle OS window events
>>   }
>>   select {
>>   case cmd := <-app.commands:
>> // process application command
>> 
>> cmd()
>>   case <-time.After(10*time.Millisecond):
>> // only wait 10ms for application commands
>> 
>>   }
>> }
>> 
>> 
>> 
>> This is an improvement, but I'd still prefer help from the Go runtime. If 
>> the runtime supported locking two goroutines to the UI thread, I could split 
>> this loop into two separate loops and remove the time.After, I *think*.
>> 
>> On Friday, January 3, 2020 at 2:59:42 PM UTC-8, robert engels wrote:
>> You can definitely run the event loop for a process on a thread other than 
>> main. The main thread is the thread created by the OS to begin running the 
>> process - the UI thread is the one that initializes the Windowing system. 
>> Some OSs even support multiple UI threads (see 
>> https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019
>>  
>> <https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019>)
>> 
>> Go doesn’t do any Windowing system initialization by default - the main 
>> thread in Go (at least according to the referenced library) is the one that 
>> called the package init() - which may not even be the OS main thread since 
>> according to the Go docs "Package initialization—variable initialization and 
>> the invocation of init functions—happens in a single goroutine, 
>> sequentially, one package at a time.” - which makes no claim that this Go 
>> routine is running on the “main” OS thread.
>> 
>> Whatever thread you use to initialize GLFW is the UI thread (or main for 
>> GLFW).
>> 
>> This is pretty standard across windowing/graphics systems.
>> 
>> 
>> 
>> 
>>> On Jan 3, 2020, at 4:02 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>>> 
>>> I'm pretty sure the OS event loop is required to be on the main thread. 
>>> 
>>> From the GLFW docs for glfwPollEvents:
>>> "This function must only be called from the main thread."
>>> 
>>> From the SDL docs for SDL_WaitEvent:
>>> "you can only call this function in the thread that initialized the video 
>>> subsystem."
>>> 
>>> 
>>> On Friday, January 3, 2020 at 1:58:29 PM UTC-8, Robert Engels wrote:
>>> Even if you could I don’t think you would want to do it this way. 
>>> 
>>> Have a go routine sleep on a channel. Post to the channel from the native 
>>> code. 
>>> 
>>> Let your command loop run on any thread and synchronize via a channel the 
>>> calls to/from native. 
>>> 
>>> The os event loop doesn’t need to run on main - it just needs to be locked 
>>> to a thread - use a native thread - and post the os events to a channel.  
>>> 
>>> Probably easiest to export a simple Go postToEventChannel() and have the 
>>> native use this. 
>>> 
>>>> On Jan 3, 2020, at 2:18 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>>>> 
>>>> 
>>>> I've been getting by with a version of this that sends commands (closures) 
>>>> to a loop on the main thread:
>>>> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55
>>>>  
>>>&g

Re: [go-nuts] Locking goroutine to "main" thread

[robert engels]

In reviewing your comments so more, I think you may be having trouble because 
you are initializing the graphics UI in the init() method. I think that is 
going to make things difficult. You are better off adding a StartUI() - which 
spawns a Go routine that handles all UI communicates (you could spawn this from 
the init() but I think that might make things harder.

You can sync the calls so only a single routine/thread will ever be created.

> On Jan 3, 2020, at 6:04 PM, robert engels  wrote:
> 
> You only need a single thread locked to the UI thread.
> 
> Use one Go routine locked to the UI thread. Put events onto a channel.
> 
> Have another Go routine read from the channel, and the app command channel 
> using select.
> 
> 
> 
>> On Jan 3, 2020, at 5:28 PM, bucha...@gmail.com <mailto:bucha...@gmail.com> 
>> wrote:
>> 
>> Whether the UI thread is on the main thread or a separate thread, a single 
>> UI thread needs to process events from two sources: the OS window and from 
>> my application. Having one loop process both sources (without support from 
>> the Go runtime scheduler) is the part I'm struggling with.
>> 
>> My latest iteration is something like:
>> 
>> for {
>>   // PollEvent doesn't block/sleep
>> 
>>   for ev := sdl.PollEvent(); ev != nil; ev = sdl.PollEvent {
>> // handle OS window events
>>   }
>>   select {
>>   case cmd := <-app.commands:
>> // process application command
>> 
>> cmd()
>>   case <-time.After(10*time.Millisecond):
>> // only wait 10ms for application commands
>> 
>>   }
>> }
>> 
>> 
>> 
>> This is an improvement, but I'd still prefer help from the Go runtime. If 
>> the runtime supported locking two goroutines to the UI thread, I could split 
>> this loop into two separate loops and remove the time.After, I *think*.
>> 
>> On Friday, January 3, 2020 at 2:59:42 PM UTC-8, robert engels wrote:
>> You can definitely run the event loop for a process on a thread other than 
>> main. The main thread is the thread created by the OS to begin running the 
>> process - the UI thread is the one that initializes the Windowing system. 
>> Some OSs even support multiple UI threads (see 
>> https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019
>>  
>> <https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019>)
>> 
>> Go doesn’t do any Windowing system initialization by default - the main 
>> thread in Go (at least according to the referenced library) is the one that 
>> called the package init() - which may not even be the OS main thread since 
>> according to the Go docs "Package initialization—variable initialization and 
>> the invocation of init functions—happens in a single goroutine, 
>> sequentially, one package at a time.” - which makes no claim that this Go 
>> routine is running on the “main” OS thread.
>> 
>> Whatever thread you use to initialize GLFW is the UI thread (or main for 
>> GLFW).
>> 
>> This is pretty standard across windowing/graphics systems.
>> 
>> 
>> 
>> 
>>> On Jan 3, 2020, at 4:02 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>>> 
>>> I'm pretty sure the OS event loop is required to be on the main thread. 
>>> 
>>> From the GLFW docs for glfwPollEvents:
>>> "This function must only be called from the main thread."
>>> 
>>> From the SDL docs for SDL_WaitEvent:
>>> "you can only call this function in the thread that initialized the video 
>>> subsystem."
>>> 
>>> 
>>> On Friday, January 3, 2020 at 1:58:29 PM UTC-8, Robert Engels wrote:
>>> Even if you could I don’t think you would want to do it this way. 
>>> 
>>> Have a go routine sleep on a channel. Post to the channel from the native 
>>> code. 
>>> 
>>> Let your command loop run on any thread and synchronize via a channel the 
>>> calls to/from native. 
>>> 
>>> The os event loop doesn’t need to run on main - it just needs to be locked 
>>> to a thread - use a native thread - and post the os events to a channel.  
>>> 
>>> Probably easiest to export a simple Go postToEventChannel() and have the 
>>> native use this. 
>>> 
>>>> On Jan 3, 2020, at 2:18 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>>>> 
>>>> 
>>>> I've 

Re: [go-nuts] Locking goroutine to "main" thread

[robert engels]

You only need a single thread locked to the UI thread.

Use one Go routine locked to the UI thread. Put events onto a channel.

Have another Go routine read from the channel, and the app command channel 
using select.



> On Jan 3, 2020, at 5:28 PM, bucha...@gmail.com wrote:
> 
> Whether the UI thread is on the main thread or a separate thread, a single UI 
> thread needs to process events from two sources: the OS window and from my 
> application. Having one loop process both sources (without support from the 
> Go runtime scheduler) is the part I'm struggling with.
> 
> My latest iteration is something like:
> 
> for {
>   // PollEvent doesn't block/sleep
> 
>   for ev := sdl.PollEvent(); ev != nil; ev = sdl.PollEvent {
> // handle OS window events
>   }
>   select {
>   case cmd := <-app.commands:
> // process application command
> 
> cmd()
>   case <-time.After(10*time.Millisecond):
> // only wait 10ms for application commands
> 
>   }
> }
> 
> 
> 
> This is an improvement, but I'd still prefer help from the Go runtime. If the 
> runtime supported locking two goroutines to the UI thread, I could split this 
> loop into two separate loops and remove the time.After, I *think*.
> 
> On Friday, January 3, 2020 at 2:59:42 PM UTC-8, robert engels wrote:
> You can definitely run the event loop for a process on a thread other than 
> main. The main thread is the thread created by the OS to begin running the 
> process - the UI thread is the one that initializes the Windowing system. 
> Some OSs even support multiple UI threads (see 
> https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019
>  
> <https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019>)
> 
> Go doesn’t do any Windowing system initialization by default - the main 
> thread in Go (at least according to the referenced library) is the one that 
> called the package init() - which may not even be the OS main thread since 
> according to the Go docs "Package initialization—variable initialization and 
> the invocation of init functions—happens in a single goroutine, sequentially, 
> one package at a time.” - which makes no claim that this Go routine is 
> running on the “main” OS thread.
> 
> Whatever thread you use to initialize GLFW is the UI thread (or main for 
> GLFW).
> 
> This is pretty standard across windowing/graphics systems.
> 
> 
> 
> 
>> On Jan 3, 2020, at 4:02 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>> 
>> I'm pretty sure the OS event loop is required to be on the main thread. 
>> 
>> From the GLFW docs for glfwPollEvents:
>> "This function must only be called from the main thread."
>> 
>> From the SDL docs for SDL_WaitEvent:
>> "you can only call this function in the thread that initialized the video 
>> subsystem."
>> 
>> 
>> On Friday, January 3, 2020 at 1:58:29 PM UTC-8, Robert Engels wrote:
>> Even if you could I don’t think you would want to do it this way. 
>> 
>> Have a go routine sleep on a channel. Post to the channel from the native 
>> code. 
>> 
>> Let your command loop run on any thread and synchronize via a channel the 
>> calls to/from native. 
>> 
>> The os event loop doesn’t need to run on main - it just needs to be locked 
>> to a thread - use a native thread - and post the os events to a channel.  
>> 
>> Probably easiest to export a simple Go postToEventChannel() and have the 
>> native use this. 
>> 
>>> On Jan 3, 2020, at 2:18 PM, buch...@ <>gmail.com <http://gmail.com/> wrote:
>>> 
>>> 
>>> I've been getting by with a version of this that sends commands (closures) 
>>> to a loop on the main thread:
>>> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55
>>>  
>>> <https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55>
>>> 
>>> And here is where it pops those commands, and also integrates with the OS 
>>> event loop:
>>> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163
>>>  
>>> <https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163>
>>> 
>>> But, I'm still not satisfied. The solution ties together the scheduling 
>>> (and code) of two separate event loops. In particular, I've found that my 
>>> commands are de

Re: [go-nuts] Locking goroutine to "main" thread

[robert engels]

You can definitely run the event loop for a process on a thread other than 
main. The main thread is the thread created by the OS to begin running the 
process - the UI thread is the one that initializes the Windowing system. Some 
OSs even support multiple UI threads (see 
https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019
 
<https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019>)

Go doesn’t do any Windowing system initialization by default - the main thread 
in Go (at least according to the referenced library) is the one that called the 
package init() - which may not even be the OS main thread since according to 
the Go docs "Package initialization—variable initialization and the invocation 
of init functions—happens in a single goroutine, sequentially, one package at a 
time.” - which makes no claim that this Go routine is running on the “main” OS 
thread.

Whatever thread you use to initialize GLFW is the UI thread (or main for GLFW).

This is pretty standard across windowing/graphics systems.




> On Jan 3, 2020, at 4:02 PM, bucha...@gmail.com wrote:
> 
> I'm pretty sure the OS event loop is required to be on the main thread. 
> 
> From the GLFW docs for glfwPollEvents:
> "This function must only be called from the main thread."
> 
> From the SDL docs for SDL_WaitEvent:
> "you can only call this function in the thread that initialized the video 
> subsystem."
> 
> 
> On Friday, January 3, 2020 at 1:58:29 PM UTC-8, Robert Engels wrote:
> Even if you could I don’t think you would want to do it this way. 
> 
> Have a go routine sleep on a channel. Post to the channel from the native 
> code. 
> 
> Let your command loop run on any thread and synchronize via a channel the 
> calls to/from native. 
> 
> The os event loop doesn’t need to run on main - it just needs to be locked to 
> a thread - use a native thread - and post the os events to a channel.  
> 
> Probably easiest to export a simple Go postToEventChannel() and have the 
> native use this. 
> 
>> On Jan 3, 2020, at 2:18 PM, buch...@gmail.com  wrote:
>> 
>> 
>> I've been getting by with a version of this that sends commands (closures) 
>> to a loop on the main thread:
>> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55
>>  
>> <https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55>
>> 
>> And here is where it pops those commands, and also integrates with the OS 
>> event loop:
>> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163
>>  
>> <https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163>
>> 
>> But, I'm still not satisfied. The solution ties together the scheduling (and 
>> code) of two separate event loops. In particular, I've found that my 
>> commands are delayed ~10ms – I think sdl.WaitEvent might be rate limited.
>> 
>> What I really want is for the two event loops (OS event loop vs app command 
>> loop) to be in two separate goroutines, both running on the main thread. 
>> That way, the OS event loop can react to infrequent window events (mouse 
>> clicks, etc) without burning lots of CPU, while my app command loop can 
>> react to commands instantly. 
>> 
>> Does that make sense? Is this possible to implement without requiring a 
>> change to the Go runtime? Is it possible to change the Go runtime to allow 
>> multiple goroutines to be scheduled only to the main thread?
>> 
>> Thanks.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/e4e3a04d-0f76-4baf-9760-9992ef38d51f%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/e4e3a04d-0f76-4baf-9760-9992ef38d51f%40googlegroups.com?utm_medium=email&utm_source=footer>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/cbbca787-1dec-4729-b91f-25

Re: [go-nuts] Locking goroutine to "main" thread

[Robert Engels]

Even if you could I don’t think you would want to do it this way. 

Have a go routine sleep on a channel. Post to the channel from the native code. 

Let your command loop run on any thread and synchronize via a channel the calls 
to/from native. 

The os event loop doesn’t need to run on main - it just needs to be locked to a 
thread - use a native thread - and post the os events to a channel.  

Probably easiest to export a simple Go postToEventChannel() and have the native 
use this. 

> On Jan 3, 2020, at 2:18 PM, bucha...@gmail.com wrote:
> 
> 
> I've been getting by with a version of this that sends commands (closures) to 
> a loop on the main thread:
> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55
> 
> And here is where it pops those commands, and also integrates with the OS 
> event loop:
> https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163
> 
> But, I'm still not satisfied. The solution ties together the scheduling (and 
> code) of two separate event loops. In particular, I've found that my commands 
> are delayed ~10ms – I think sdl.WaitEvent might be rate limited.
> 
> What I really want is for the two event loops (OS event loop vs app command 
> loop) to be in two separate goroutines, both running on the main thread. That 
> way, the OS event loop can react to infrequent window events (mouse clicks, 
> etc) without burning lots of CPU, while my app command loop can react to 
> commands instantly. 
> 
> Does that make sense? Is this possible to implement without requiring a 
> change to the Go runtime? Is it possible to change the Go runtime to allow 
> multiple goroutines to be scheduled only to the main thread?
> 
> Thanks.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/e4e3a04d-0f76-4baf-9760-9992ef38d51f%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/FFE5D51B-757C-4045-8C70-A764D32E9CEA%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

Also, you might find something like nats.io simplifies you’re effort. I’m not 
sure if it supports extremely large message sizes off hand, but there is 
probably a chunking layer available. Doing simple TCP messaging is fairly easy, 
when you get into redundancy, fan-out, etc is can get complex fast, so using a 
messaging library may help. 

> On Dec 31, 2019, at 10:37 AM, Robert Engels  wrote:
> 
> 
> All of the source is there... but in general it is a bit more complex under 
> the covers than most - as it uses select/poll to know when a socket is ready 
> for IO then schedules the routine in the Read() to perform the IO. So Go has 
> a bit of its own kernel code than you would see in typical synchronous C 
> socket code. Ultimately Read() becomes a standard read() on the socket 
> descriptor. 
> 
>>> On Dec 31, 2019, at 10:03 AM, Ron Wahler  wrote:
>>> 
>> 
>> \\One other note, if you have a request / response type protocol with fairly 
>> defined lengths, you don’t need a buffer larger than the largest message if 
>> you don’t allow concurrent requests from the same client. 
>> 
>> Yes, understood, that was not the constraint, I have to process an unknown 
>> amount of bytes, so I have been just trying to make sure I had the best 
>> solution.  
>> 
>> I would still like to read the underlying GoLang code on Read() to see what 
>> assumptions it makes to the system calls.
>> 
>> thanks,
>> Ron
>> 
>>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>>> I am looking for a net.conn standard read that would return a data buffer 
>>> the exact size of the read. I am trying to read an unknown amount of byte 
>>> data from the connection. With the read i am using I am required to 
>>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>>> read that works more like  the ReadString , but is for a byte slice.
>>> 
>>> // I want something similar to this read that returns the read string into 
>>> the message string.
>>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>>> 
>>> if ( err != nil ){
>>> 
>>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>>> err)
>>> 
>>> return 
>>> 
>>> }
>>> 
>>> 
>>> 
>>> 
>>> 
>>> // had to preallocate a buffer, but I want a read to return me a buffer so 
>>> I don't have to guess how big to make it.
>>> 
>>>  buf := make([]byte, 1024*32)
>>> 
>>>  // READ FROM CLIENT
>>> 
>>>  nBytes, err := Csrc.Read(buf)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Is this not possible, I have not seen any examples that would indicate that 
>>> there is a standard library that would do something like what I am looking 
>>> for.
>>> 
>>> 
>>> 
>>> thanks,
>>> 
>>> Ron
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/24a6f6bd-8948-434c-bb83-c663d75bb23d%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/113814DD-0320-4B19-ADB4-21E6ECF553BC%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

All of the source is there... but in general it is a bit more complex under the 
covers than most - as it uses select/poll to know when a socket is ready for IO 
then schedules the routine in the Read() to perform the IO. So Go has a bit of 
its own kernel code than you would see in typical synchronous C socket code. 
Ultimately Read() becomes a standard read() on the socket descriptor. 

> On Dec 31, 2019, at 10:03 AM, Ron Wahler  wrote:
> 
> 
> \\One other note, if you have a request / response type protocol with fairly 
> defined lengths, you don’t need a buffer larger than the largest message if 
> you don’t allow concurrent requests from the same client. 
> 
> Yes, understood, that was not the constraint, I have to process an unknown 
> amount of bytes, so I have been just trying to make sure I had the best 
> solution.  
> 
> I would still like to read the underlying GoLang code on Read() to see what 
> assumptions it makes to the system calls.
> 
> thanks,
> Ron
> 
>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>> I am looking for a net.conn standard read that would return a data buffer 
>> the exact size of the read. I am trying to read an unknown amount of byte 
>> data from the connection. With the read i am using I am required to 
>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>> read that works more like  the ReadString , but is for a byte slice.
>> 
>> // I want something similar to this read that returns the read string into 
>> the message string.
>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>> 
>> if ( err != nil ){
>> 
>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>> err)
>> 
>> return 
>> 
>> }
>> 
>> 
>> 
>> 
>> 
>> // had to preallocate a buffer, but I want a read to return me a buffer so I 
>> don't have to guess how big to make it.
>> 
>>  buf := make([]byte, 1024*32)
>> 
>>  // READ FROM CLIENT
>> 
>>  nBytes, err := Csrc.Read(buf)
>> 
>> 
>> 
>> 
>> 
>> Is this not possible, I have not seen any examples that would indicate that 
>> there is a standard library that would do something like what I am looking 
>> for.
>> 
>> 
>> 
>> thanks,
>> 
>> Ron
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/24a6f6bd-8948-434c-bb83-c663d75bb23d%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CFE462A5-C276-4B13-BE6B-6057A1B8DC3F%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

One other note, if you have a request / response type protocol with fairly 
defined lengths, you don’t need a buffer larger than the largest message if you 
don’t allow concurrent requests from the same client. 

> On Dec 31, 2019, at 9:35 AM, Robert Engels  wrote:
> 
> 
> The important factors: 1) the larger the buffer the fewer system calls that 
> need to be made 2) the larger the buffer the more memory use which can be 
> significant with many simultaneous connections
> 
> You need to remember that with correct socket options the kernel is already 
> buffering, and adding your own buffering on top may not be needed 3) really 
> large user space (Go) buffers may be needed if processing a really fast fat 
> pipe
> 
>>> On Dec 31, 2019, at 9:23 AM, Ron Wahler  wrote:
>>> 
>> 
>> Thanks for all the great responses.
>> 
>> How much of the read behavior is in the golang underlying code on the read 
>> and how much is 
>> on the underlying OS driver that implements the behavior of the read. I 
>> understand the stream nature of the TCP connection and I handle that in my 
>> code, I was just looking to optimize the buffer allocation for the read on 
>> each packet  of the stream so I don't have to over allocate the buf if I 
>> scaled out my function to be concurrent. From the responses so far I think I 
>> just have to pick a buffer size for the Read() or the ReadAtLeast() and just 
>> process multiple buffers with my pre-allocated guess on the buffer size as I 
>> am already doing.
>> 
>> I was also looking to understand the read implementation of GoLand and how 
>> it utilizes the underlying drivers, so any info on that or where to read the 
>> code for golang integration with the drivers would be great.
>> 
>> cheers,
>> Ron
>> 
>>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>>> I am looking for a net.conn standard read that would return a data buffer 
>>> the exact size of the read. I am trying to read an unknown amount of byte 
>>> data from the connection. With the read i am using I am required to 
>>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>>> read that works more like  the ReadString , but is for a byte slice.
>>> 
>>> // I want something similar to this read that returns the read string into 
>>> the message string.
>>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>>> 
>>> if ( err != nil ){
>>> 
>>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>>> err)
>>> 
>>> return 
>>> 
>>> }
>>> 
>>> 
>>> 
>>> 
>>> 
>>> // had to preallocate a buffer, but I want a read to return me a buffer so 
>>> I don't have to guess how big to make it.
>>> 
>>>  buf := make([]byte, 1024*32)
>>> 
>>>  // READ FROM CLIENT
>>> 
>>>  nBytes, err := Csrc.Read(buf)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Is this not possible, I have not seen any examples that would indicate that 
>>> there is a standard library that would do something like what I am looking 
>>> for.
>>> 
>>> 
>>> 
>>> thanks,
>>> 
>>> Ron
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/0e735f0e-dfb1-4e1f-9a2c-1b153be20514%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/DA1AEA55-8C9D-4E19-860E-A6FBA417406A%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

The important factors: 1) the larger the buffer the fewer system calls that 
need to be made 2) the larger the buffer the more memory use which can be 
significant with many simultaneous connections

You need to remember that with correct socket options the kernel is already 
buffering, and adding your own buffering on top may not be needed 3) really 
large user space (Go) buffers may be needed if processing a really fast fat pipe

> On Dec 31, 2019, at 9:23 AM, Ron Wahler  wrote:
> 
> 
> Thanks for all the great responses.
> 
> How much of the read behavior is in the golang underlying code on the read 
> and how much is 
> on the underlying OS driver that implements the behavior of the read. I 
> understand the stream nature of the TCP connection and I handle that in my 
> code, I was just looking to optimize the buffer allocation for the read on 
> each packet  of the stream so I don't have to over allocate the buf if I 
> scaled out my function to be concurrent. From the responses so far I think I 
> just have to pick a buffer size for the Read() or the ReadAtLeast() and just 
> process multiple buffers with my pre-allocated guess on the buffer size as I 
> am already doing.
> 
> I was also looking to understand the read implementation of GoLand and how it 
> utilizes the underlying drivers, so any info on that or where to read the 
> code for golang integration with the drivers would be great.
> 
> cheers,
> Ron
> 
>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>> I am looking for a net.conn standard read that would return a data buffer 
>> the exact size of the read. I am trying to read an unknown amount of byte 
>> data from the connection. With the read i am using I am required to 
>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>> read that works more like  the ReadString , but is for a byte slice.
>> 
>> // I want something similar to this read that returns the read string into 
>> the message string.
>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>> 
>> if ( err != nil ){
>> 
>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>> err)
>> 
>> return 
>> 
>> }
>> 
>> 
>> 
>> 
>> 
>> // had to preallocate a buffer, but I want a read to return me a buffer so I 
>> don't have to guess how big to make it.
>> 
>>  buf := make([]byte, 1024*32)
>> 
>>  // READ FROM CLIENT
>> 
>>  nBytes, err := Csrc.Read(buf)
>> 
>> 
>> 
>> 
>> 
>> Is this not possible, I have not seen any examples that would indicate that 
>> there is a standard library that would do something like what I am looking 
>> for.
>> 
>> 
>> 
>> thanks,
>> 
>> Ron
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0e735f0e-dfb1-4e1f-9a2c-1b153be20514%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/060D2B13-71A9-42AA-9002-0924E1A051CB%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

Oh, and I don’t think SCTP is natively supported on Windows yet. 

So your interoperability May vary...

> On Dec 30, 2019, at 4:17 PM, Robert Engels  wrote:
> 
> 
> Im pretty sure I’m correct. It is a socket type not an option on TCP, which 
> equates to a different protocol. If you use that option you get a SCTP 
> transport not TCP. 
> 
>>> On Dec 30, 2019, at 4:06 PM, Bruno Albuquerque  wrote:
>>> 
>> 
>> Although I am no expert in the subject, I would doubt this assertion. It is 
>> there in the socket man page in a Ubuntu machine with no mention of anything 
>> specific being needed (other than the implicit fact that you need a TCP 
>> stack that supports it, which should be true for any modern version of Linux 
>> anyway).
>> 
>> 
>>> On Mon, Dec 30, 2019 at 12:23 PM Robert Engels  
>>> wrote:
>>> That option requires proprietary protocols not standard tcp/udp. 
>>> 
>>>>> On Dec 30, 2019, at 12:04 PM, Bruno Albuquerque  wrote:
>>>>> 
>>>> 
>>>> But, to complicate things, you can create what is basically a TCp 
>>>> connection with packet boundaries using SOCK_SEQPACKET (as opposed to 
>>>> SOCK_STREAM or SOCK_DGRAM).
>>>> 
>>>>> On Mon, Dec 30, 2019 at 9:04 AM Jake Montgomery  
>>>>> wrote:
>>>>> It sounds like maybe you have some misconceptions about TCP. It is a 
>>>>> stream protocol, there are no data boundaries that are preserved. If send 
>>>>> 20 bytes via TCP in a single call, it is likely that those 20 will arrive 
>>>>> together at the client. But it is NOT guaranteed. It is perfectly 
>>>>> legitimate for 10 bytes to arrive first, then the next 10 sometime later. 
>>>>> Obviously this is unlikely with only a few bytes, but becomes more likely 
>>>>> as the size of the Write grows. Until the connection is closed, you never 
>>>>> know if there is more data coming. So it may seem that there is a 1:1 
>>>>> correlation between conn.Write() and conn.Read(), but you can not count 
>>>>> on it. 
>>>>> 
>>>>> To answer you specific question, conn.Read() will return when it has 
>>>>> filled up the buffer provided, or there is no more data ready to be read 
>>>>> at that moment. ReadAll() will wait until EOF. Given that TCP is a 
>>>>> stream, as I described above, it is still unclear what you hope to have 
>>>>> happen without knowing more about the specific data being transmitted, 
>>>>> and what you wan to do with it on the client side. 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Sunday, December 29, 2019 at 10:20:39 AM UTC-5, Ron Wahler wrote:
>>>>>> Jake,
>>>>>> 
>>>>>> Thanks for the reply. Csrc.Read is what I was referring to as the 
>>>>>> connection standard read, should not have used the words "standard read" 
>>>>>> sorry about that. The problem I am trying to solve is reading an unknown 
>>>>>> amount of byte data.  I am trying to understand what triggers the 
>>>>>> Csrc.Read(buf) to return when I send say 3 bytes to it with a client, I 
>>>>>> also keep the connection open and send a few bytes of characters with 
>>>>>> the netcat tool, the Csrc.Read returns, but the snip it below that with 
>>>>>> ReadAll does not return. I am trying to understand the underlying 
>>>>>> behavior of what triggers a return with the data in these two calls ?
>>>>>> 
>>>>>> on this read :
>>>>>> 
>>>>>> Csrc net.Conn
>>>>>> 
>>>>>> 
>>>>>>  buf := make([]byte, 1024*32)
>>>>>> 
>>>>>>  // READ FROM CLIENT
>>>>>> 
>>>>>>  nBytes, err := Csrc.Read(buf)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Csrc.Read(buf)  returns with a few bytes that I send to it.  It does not 
>>>>>> wait for the entire allocated buf size to return. This works great, but 
>>>>>> I am looking for a way to not preallocate a large buffer.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I am prototyping with ReadAll, see the following snip it, but when I 
>>>>>> send a few

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

Im pretty sure I’m correct. It is a socket type not an option on TCP, which 
equates to a different protocol. If you use that option you get a SCTP 
transport not TCP. 

> On Dec 30, 2019, at 4:06 PM, Bruno Albuquerque  wrote:
> 
> 
> Although I am no expert in the subject, I would doubt this assertion. It is 
> there in the socket man page in a Ubuntu machine with no mention of anything 
> specific being needed (other than the implicit fact that you need a TCP stack 
> that supports it, which should be true for any modern version of Linux 
> anyway).
> 
> 
>> On Mon, Dec 30, 2019 at 12:23 PM Robert Engels  wrote:
>> That option requires proprietary protocols not standard tcp/udp. 
>> 
>>>> On Dec 30, 2019, at 12:04 PM, Bruno Albuquerque  wrote:
>>>> 
>>> 
>>> But, to complicate things, you can create what is basically a TCp 
>>> connection with packet boundaries using SOCK_SEQPACKET (as opposed to 
>>> SOCK_STREAM or SOCK_DGRAM).
>>> 
>>>> On Mon, Dec 30, 2019 at 9:04 AM Jake Montgomery  wrote:
>>>> It sounds like maybe you have some misconceptions about TCP. It is a 
>>>> stream protocol, there are no data boundaries that are preserved. If send 
>>>> 20 bytes via TCP in a single call, it is likely that those 20 will arrive 
>>>> together at the client. But it is NOT guaranteed. It is perfectly 
>>>> legitimate for 10 bytes to arrive first, then the next 10 sometime later. 
>>>> Obviously this is unlikely with only a few bytes, but becomes more likely 
>>>> as the size of the Write grows. Until the connection is closed, you never 
>>>> know if there is more data coming. So it may seem that there is a 1:1 
>>>> correlation between conn.Write() and conn.Read(), but you can not count on 
>>>> it. 
>>>> 
>>>> To answer you specific question, conn.Read() will return when it has 
>>>> filled up the buffer provided, or there is no more data ready to be read 
>>>> at that moment. ReadAll() will wait until EOF. Given that TCP is a stream, 
>>>> as I described above, it is still unclear what you hope to have happen 
>>>> without knowing more about the specific data being transmitted, and what 
>>>> you wan to do with it on the client side. 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> On Sunday, December 29, 2019 at 10:20:39 AM UTC-5, Ron Wahler wrote:
>>>>> Jake,
>>>>> 
>>>>> Thanks for the reply. Csrc.Read is what I was referring to as the 
>>>>> connection standard read, should not have used the words "standard read" 
>>>>> sorry about that. The problem I am trying to solve is reading an unknown 
>>>>> amount of byte data.  I am trying to understand what triggers the 
>>>>> Csrc.Read(buf) to return when I send say 3 bytes to it with a client, I 
>>>>> also keep the connection open and send a few bytes of characters with the 
>>>>> netcat tool, the Csrc.Read returns, but the snip it below that with 
>>>>> ReadAll does not return. I am trying to understand the underlying 
>>>>> behavior of what triggers a return with the data in these two calls ?
>>>>> 
>>>>> on this read :
>>>>> 
>>>>> Csrc net.Conn
>>>>> 
>>>>> 
>>>>>  buf := make([]byte, 1024*32)
>>>>> 
>>>>>  // READ FROM CLIENT
>>>>> 
>>>>>  nBytes, err := Csrc.Read(buf)
>>>>> 
>>>>> 
>>>>> 
>>>>> Csrc.Read(buf)  returns with a few bytes that I send to it.  It does not 
>>>>> wait for the entire allocated buf size to return. This works great, but I 
>>>>> am looking for a way to not preallocate a large buffer.
>>>>> 
>>>>> 
>>>>> 
>>>>> I am prototyping with ReadAll, see the following snip it, but when I send 
>>>>> a few bytes to this call with a client, it does not return. The 
>>>>> documentation is saying it may be  looking for an EOF which I do not send.
>>>>> 
>>>>> 
>>>>> 
>>>>>   buf, read_err := ioutil.ReadAll(Csrc)
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> thanks,
>>>>> 
>>>>> Ron
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>&

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

I think that distinction is splitting hairs a bit in the case of Go - usually when you speak of concurrent you are talking about completely disparate processes (in the context of an OS). A typically Go server might be handling many types of client requests but it can easily be viewed as it is parallelizing the "all the clients work" - the Go program runs concurrently with other OS programs, but is parallelizing its own work.-Original Message-
From: Jesper Louis Andersen 
Sent: Dec 30, 2019 3:41 PM
To: Robert Engels 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

On Mon, Dec 30, 2019 at 10:14 PM Robert Engels <reng...@ix.netcom.com> wrote:Here is a simple test that demonstrates the dynamics https://play.golang.org/p/6SZcxCEAfFp (cannot run in playground)Notice that the call that uses an over allocated number of routines takes 5x longer wall time than the properly sized one - this is due to scheduling and contention on the underlying structures. So blindly creating go routines does not achieve optimum performance for many workloads (even when the number of OS threads is capped).rengels@rengels:~/gotest$ go run main_bench.go 2.261805812s1.311269725s6.341378965sYes, this is unsurprising since the problem in the program is parallel.In a typical concurrent problem, most work is sitting and waiting for some event to happen and those events are usually measured in several milliseconds. For those problems, the switching cost is very low given the other benefits you gain. The key property there is rarely about speed, but about description of the problem to the machine.Go definitely leans toward concurrent problems. If you want to target parallel problems, you have other options which are better, depending on factors of distribution, GPU availability, etc.-- J.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1562530823.2748.1577742659901%40wamui-sophie.atl.sa.earthlink.net.

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

One thing this exercise also reminded me of, is that using Go for any sort of "real time signal processing" is going to be very difficult - maybe if you lock the event handling routine to a thread, then use native code to change the thread priority - not sure how that would interact with the Go scheduler/GC, otherwise the "handler" is the same priority as all other Go routines, which makes it critical to not over-allocate the total number of routines running (vs available cores).Probably to do this properly you need buffered hardware and kernel support / native code (but signal handling is a catch all, could be reading from a high-priority socket, etc.)-Original Message-
From: Robert Engels 
Sent: Dec 30, 2019 3:21 PM
To: Robert Engels , Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Also, if running on a machine with a low cpu count (gomaxprocs) you probably need to increase the 'total' multiplier (mine was 12).-----Original Message-
From: Robert Engels 
Sent: Dec 30, 2019 3:14 PM
To: Robert Engels , Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Here is a simple test that demonstrates the dynamics https://play.golang.org/p/6SZcxCEAfFp (cannot run in playground)Notice that the call that uses an over allocated number of routines takes 5x longer wall time than the properly sized one - this is due to scheduling and contention on the underlying structures. So blindly creating go routines does not achieve optimum performance for many workloads (even when the number of OS threads is capped).rengels@rengels:~/gotest$ go run main_bench.go 2.261805812s1.311269725s6.341378965s-----Original Message-
From: Robert Engels 
Sent: Dec 30, 2019 9:43 AM
To: Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Right, but the overhead is not constant nor free. So if you parallelize the CPU bound  task into 100 segments and you only have 10 cores, the contention on the internal locking structures (scheduler, locks in channels) will be significant and the entire process will probably take far longer - working on a simple test case to demonstrate - so blindly spinning up Go routines will not be the best solution for some workloads. On Dec 30, 2019, at 9:11 AM, Jesper Louis Andersen  wrote:On Mon, Dec 30, 2019 at 10:46 AM Brian Candler <b.cand...@pobox.com> wrote:Which switching cost are you referring to?  The switching cost between goroutines? This is minimal, as it takes places within a single thread.  Or are you referring to cache invalidation issues?  Or something else?



It is the usual dichotomy between concurrency and parallelism. If your problem is concurrent, then it is usually the case that the switching cost is minimal and it is often far easier just to run thousands of goroutines on a much smaller set of cores. However, if your problem is parallel, you often have one task which is your goal, and you want it to finish as soon as possible[0]. Here, explicit control over the cores tend to be more efficient due to caches, memory bandwidth, latency hiding, etc. Processor affinity and pinning threads to processors is often important in this game. But unless you can gain a significant speedup by doing this explicitly, I wouldn't bother.[0] There is another, sometimes better, way to look at the problem. Rather than being able to solve a problem N times faster, you can also see at as being able to solve a problem N times larger in the same time. This has the advantage that communication is less of a problem. When the problem size gets small enough, the overhead of multiple processing cores gets too large.-- J.



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGrdgiVje8SyiNZG-5JE8gE-%3DYOyDjdvVn-uyy95P3zzepc3wA%40mail.gmail.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/BA8435BE-3D84-446D-8C82-66A728A3F9AE%40ix.netcom.com.





-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/408047985.1927.1577740873451%40wamui-lola.atl.sa.earthlink.net.




-- 
You received this message 

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

Also, if running on a machine with a low cpu count (gomaxprocs) you probably need to increase the 'total' multiplier (mine was 12).-Original Message-----
From: Robert Engels 
Sent: Dec 30, 2019 3:14 PM
To: Robert Engels , Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Here is a simple test that demonstrates the dynamics https://play.golang.org/p/6SZcxCEAfFp (cannot run in playground)Notice that the call that uses an over allocated number of routines takes 5x longer wall time than the properly sized one - this is due to scheduling and contention on the underlying structures. So blindly creating go routines does not achieve optimum performance for many workloads (even when the number of OS threads is capped).rengels@rengels:~/gotest$ go run main_bench.go 2.261805812s1.311269725s6.341378965s-Original Message-----
From: Robert Engels 
Sent: Dec 30, 2019 9:43 AM
To: Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Right, but the overhead is not constant nor free. So if you parallelize the CPU bound  task into 100 segments and you only have 10 cores, the contention on the internal locking structures (scheduler, locks in channels) will be significant and the entire process will probably take far longer - working on a simple test case to demonstrate - so blindly spinning up Go routines will not be the best solution for some workloads. On Dec 30, 2019, at 9:11 AM, Jesper Louis Andersen  wrote:On Mon, Dec 30, 2019 at 10:46 AM Brian Candler <b.cand...@pobox.com> wrote:Which switching cost are you referring to?  The switching cost between goroutines? This is minimal, as it takes places within a single thread.  Or are you referring to cache invalidation issues?  Or something else?



It is the usual dichotomy between concurrency and parallelism. If your problem is concurrent, then it is usually the case that the switching cost is minimal and it is often far easier just to run thousands of goroutines on a much smaller set of cores. However, if your problem is parallel, you often have one task which is your goal, and you want it to finish as soon as possible[0]. Here, explicit control over the cores tend to be more efficient due to caches, memory bandwidth, latency hiding, etc. Processor affinity and pinning threads to processors is often important in this game. But unless you can gain a significant speedup by doing this explicitly, I wouldn't bother.[0] There is another, sometimes better, way to look at the problem. Rather than being able to solve a problem N times faster, you can also see at as being able to solve a problem N times larger in the same time. This has the advantage that communication is less of a problem. When the problem size gets small enough, the overhead of multiple processing cores gets too large.-- J.



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGrdgiVje8SyiNZG-5JE8gE-%3DYOyDjdvVn-uyy95P3zzepc3wA%40mail.gmail.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/BA8435BE-3D84-446D-8C82-66A728A3F9AE%40ix.netcom.com.





-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/408047985.1927.1577740873451%40wamui-lola.atl.sa.earthlink.net.

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

Here is a simple test that demonstrates the dynamics https://play.golang.org/p/6SZcxCEAfFp (cannot run in playground)Notice that the call that uses an over allocated number of routines takes 5x longer wall time than the properly sized one - this is due to scheduling and contention on the underlying structures. So blindly creating go routines does not achieve optimum performance for many workloads (even when the number of OS threads is capped).rengels@rengels:~/gotest$ go run main_bench.go 2.261805812s1.311269725s6.341378965s-Original Message-
From: Robert Engels 
Sent: Dec 30, 2019 9:43 AM
To: Jesper Louis Andersen 
Cc: Brian Candler , golang-nuts 
Subject: Re: [go-nuts] Simple worker pool in golnag

Right, but the overhead is not constant nor free. So if you parallelize the CPU bound  task into 100 segments and you only have 10 cores, the contention on the internal locking structures (scheduler, locks in channels) will be significant and the entire process will probably take far longer - working on a simple test case to demonstrate - so blindly spinning up Go routines will not be the best solution for some workloads. On Dec 30, 2019, at 9:11 AM, Jesper Louis Andersen  wrote:On Mon, Dec 30, 2019 at 10:46 AM Brian Candler <b.cand...@pobox.com> wrote:Which switching cost are you referring to?  The switching cost between goroutines? This is minimal, as it takes places within a single thread.  Or are you referring to cache invalidation issues?  Or something else?



It is the usual dichotomy between concurrency and parallelism. If your problem is concurrent, then it is usually the case that the switching cost is minimal and it is often far easier just to run thousands of goroutines on a much smaller set of cores. However, if your problem is parallel, you often have one task which is your goal, and you want it to finish as soon as possible[0]. Here, explicit control over the cores tend to be more efficient due to caches, memory bandwidth, latency hiding, etc. Processor affinity and pinning threads to processors is often important in this game. But unless you can gain a significant speedup by doing this explicitly, I wouldn't bother.[0] There is another, sometimes better, way to look at the problem. Rather than being able to solve a problem N times faster, you can also see at as being able to solve a problem N times larger in the same time. This has the advantage that communication is less of a problem. When the problem size gets small enough, the overhead of multiple processing cores gets too large.-- J.



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGrdgiVje8SyiNZG-5JE8gE-%3DYOyDjdvVn-uyy95P3zzepc3wA%40mail.gmail.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/BA8435BE-3D84-446D-8C82-66A728A3F9AE%40ix.netcom.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/740063269.1900.1577740472713%40wamui-lola.atl.sa.earthlink.net.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

That option requires proprietary protocols not standard tcp/udp. 

> On Dec 30, 2019, at 12:04 PM, Bruno Albuquerque  wrote:
> 
> 
> But, to complicate things, you can create what is basically a TCp connection 
> with packet boundaries using SOCK_SEQPACKET (as opposed to SOCK_STREAM or 
> SOCK_DGRAM).
> 
>> On Mon, Dec 30, 2019 at 9:04 AM Jake Montgomery  wrote:
>> It sounds like maybe you have some misconceptions about TCP. It is a stream 
>> protocol, there are no data boundaries that are preserved. If send 20 bytes 
>> via TCP in a single call, it is likely that those 20 will arrive together at 
>> the client. But it is NOT guaranteed. It is perfectly legitimate for 10 
>> bytes to arrive first, then the next 10 sometime later. Obviously this is 
>> unlikely with only a few bytes, but becomes more likely as the size of the 
>> Write grows. Until the connection is closed, you never know if there is more 
>> data coming. So it may seem that there is a 1:1 correlation between 
>> conn.Write() and conn.Read(), but you can not count on it. 
>> 
>> To answer you specific question, conn.Read() will return when it has filled 
>> up the buffer provided, or there is no more data ready to be read at that 
>> moment. ReadAll() will wait until EOF. Given that TCP is a stream, as I 
>> described above, it is still unclear what you hope to have happen without 
>> knowing more about the specific data being transmitted, and what you wan to 
>> do with it on the client side. 
>> 
>> 
>> 
>> 
>>> On Sunday, December 29, 2019 at 10:20:39 AM UTC-5, Ron Wahler wrote:
>>> Jake,
>>> 
>>> Thanks for the reply. Csrc.Read is what I was referring to as the 
>>> connection standard read, should not have used the words "standard read" 
>>> sorry about that. The problem I am trying to solve is reading an unknown 
>>> amount of byte data.  I am trying to understand what triggers the 
>>> Csrc.Read(buf) to return when I send say 3 bytes to it with a client, I 
>>> also keep the connection open and send a few bytes of characters with the 
>>> netcat tool, the Csrc.Read returns, but the snip it below that with ReadAll 
>>> does not return. I am trying to understand the underlying behavior of what 
>>> triggers a return with the data in these two calls ?
>>> 
>>> on this read :
>>> 
>>> Csrc net.Conn
>>> 
>>> 
>>>  buf := make([]byte, 1024*32)
>>> 
>>>  // READ FROM CLIENT
>>> 
>>>  nBytes, err := Csrc.Read(buf)
>>> 
>>> 
>>> 
>>> Csrc.Read(buf)  returns with a few bytes that I send to it.  It does not 
>>> wait for the entire allocated buf size to return. This works great, but I 
>>> am looking for a way to not preallocate a large buffer.
>>> 
>>> 
>>> 
>>> I am prototyping with ReadAll, see the following snip it, but when I send a 
>>> few bytes to this call with a client, it does not return. The documentation 
>>> is saying it may be  looking for an EOF which I do not send.
>>> 
>>> 
>>> 
>>>   buf, read_err := ioutil.ReadAll(Csrc)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> thanks,
>>> 
>>> Ron
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
 On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
 I am looking for a net.conn standard read that would return a data buffer 
 the exact size of the read. I am trying to read an unknown amount of byte 
 data from the connection. With the read i am using I am required to 
 pre-allocate a buffer and pass that buffer to the read. I am looking for a 
 read that works more like  the ReadString , but is for a byte slice.
 
 // I want something similar to this read that returns the read string into 
 the message string.
  message, err := bufio.NewReader(ServerConn).ReadString('\n')
 
 if ( err != nil ){
 
 fmt.Println("RELAY: ERROR:  Reg Message read 
 err:", err)
 
 return 
 
 }
 
 
 
 
 
 // had to preallocate a buffer, but I want a read to return me a buffer so 
 I don't have to guess how big to make it.
 
  buf := make([]byte, 1024*32)
 
  // READ FROM CLIENT
 
  nBytes, err := Csrc.Read(buf)
 
 
 
 
 
 Is this not possible, I have not seen any examples that would indicate 
 that there is a standard library that would do something like what I am 
 looking for.
 
 
 
 thanks,
 
 Ron
 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/7e468329-2488-460c-9419-b4c55857b1eb%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

ReadAll reads until buffer length or EOF. 

> On Dec 30, 2019, at 11:04 AM, Jake Montgomery  wrote:
> 
> 
> It sounds like maybe you have some misconceptions about TCP. It is a stream 
> protocol, there are no data boundaries that are preserved. If send 20 bytes 
> via TCP in a single call, it is likely that those 20 will arrive together at 
> the client. But it is NOT guaranteed. It is perfectly legitimate for 10 bytes 
> to arrive first, then the next 10 sometime later. Obviously this is unlikely 
> with only a few bytes, but becomes more likely as the size of the Write 
> grows. Until the connection is closed, you never know if there is more data 
> coming. So it may seem that there is a 1:1 correlation between conn.Write() 
> and conn.Read(), but you can not count on it. 
> 
> To answer you specific question, conn.Read() will return when it has filled 
> up the buffer provided, or there is no more data ready to be read at that 
> moment. ReadAll() will wait until EOF. Given that TCP is a stream, as I 
> described above, it is still unclear what you hope to have happen without 
> knowing more about the specific data being transmitted, and what you wan to 
> do with it on the client side. 
> 
> 
> 
> 
>> On Sunday, December 29, 2019 at 10:20:39 AM UTC-5, Ron Wahler wrote:
>> Jake,
>> 
>> Thanks for the reply. Csrc.Read is what I was referring to as the connection 
>> standard read, should not have used the words "standard read" sorry about 
>> that. The problem I am trying to solve is reading an unknown amount of byte 
>> data.  I am trying to understand what triggers the Csrc.Read(buf) to return 
>> when I send say 3 bytes to it with a client, I also keep the connection open 
>> and send a few bytes of characters with the netcat tool, the Csrc.Read 
>> returns, but the snip it below that with ReadAll does not return. I am 
>> trying to understand the underlying behavior of what triggers a return with 
>> the data in these two calls ?
>> 
>> on this read :
>> 
>> Csrc net.Conn
>> 
>> 
>>  buf := make([]byte, 1024*32)
>> 
>>  // READ FROM CLIENT
>> 
>>  nBytes, err := Csrc.Read(buf)
>> 
>> 
>> 
>> Csrc.Read(buf)  returns with a few bytes that I send to it.  It does not 
>> wait for the entire allocated buf size to return. This works great, but I am 
>> looking for a way to not preallocate a large buffer.
>> 
>> 
>> 
>> I am prototyping with ReadAll, see the following snip it, but when I send a 
>> few bytes to this call with a client, it does not return. The documentation 
>> is saying it may be  looking for an EOF which I do not send.
>> 
>> 
>> 
>>   buf, read_err := ioutil.ReadAll(Csrc)
>> 
>> 
>> 
>> 
>> 
>> thanks,
>> 
>> Ron
>> 
>> 
>> 
>> 
>> 
>> 
>>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>>> I am looking for a net.conn standard read that would return a data buffer 
>>> the exact size of the read. I am trying to read an unknown amount of byte 
>>> data from the connection. With the read i am using I am required to 
>>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>>> read that works more like  the ReadString , but is for a byte slice.
>>> 
>>> // I want something similar to this read that returns the read string into 
>>> the message string.
>>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>>> 
>>> if ( err != nil ){
>>> 
>>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>>> err)
>>> 
>>> return 
>>> 
>>> }
>>> 
>>> 
>>> 
>>> 
>>> 
>>> // had to preallocate a buffer, but I want a read to return me a buffer so 
>>> I don't have to guess how big to make it.
>>> 
>>>  buf := make([]byte, 1024*32)
>>> 
>>>  // READ FROM CLIENT
>>> 
>>>  nBytes, err := Csrc.Read(buf)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Is this not possible, I have not seen any examples that would indicate that 
>>> there is a standard library that would do something like what I am looking 
>>> for.
>>> 
>>> 
>>> 
>>> thanks,
>>> 
>>> Ron
>>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/7e468329-2488-460c-9419-b4c55857b1eb%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C505C73A-325C-4AEA-A564-326590AD97E4%40ix.netcom.com.

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

Right, but the overhead is not constant nor free. So if you parallelize the CPU 
bound  task into 100 segments and you only have 10 cores, the contention on the 
internal locking structures (scheduler, locks in channels) will be significant 
and the entire process will probably take far longer - working on a simple test 
case to demonstrate - so blindly spinning up Go routines will not be the best 
solution for some workloads. 

> On Dec 30, 2019, at 9:11 AM, Jesper Louis Andersen 
>  wrote:
> 
> 
>> On Mon, Dec 30, 2019 at 10:46 AM Brian Candler  wrote:
>> Which switching cost are you referring to?  The switching cost between 
>> goroutines? This is minimal, as it takes places within a single thread.  Or 
>> are you referring to cache invalidation issues?  Or something else?
> 
> It is the usual dichotomy between concurrency and parallelism. If your 
> problem is concurrent, then it is usually the case that the switching cost is 
> minimal and it is often far easier just to run thousands of goroutines on a 
> much smaller set of cores. However, if your problem is parallel, you often 
> have one task which is your goal, and you want it to finish as soon as 
> possible[0]. Here, explicit control over the cores tend to be more efficient 
> due to caches, memory bandwidth, latency hiding, etc. Processor affinity and 
> pinning threads to processors is often important in this game. But unless you 
> can gain a significant speedup by doing this explicitly, I wouldn't bother.
> 
> [0] There is another, sometimes better, way to look at the problem. Rather 
> than being able to solve a problem N times faster, you can also see at as 
> being able to solve a problem N times larger in the same time. This has the 
> advantage that communication is less of a problem. When the problem size gets 
> small enough, the overhead of multiple processing cores gets too large.
> 
> 
> -- 
> J.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAGrdgiVje8SyiNZG-5JE8gE-%3DYOyDjdvVn-uyy95P3zzepc3wA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/BA8435BE-3D84-446D-8C82-66A728A3F9AE%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

Use read and expand the buffer as needed (or write the chunks to a file). If at 
the end it is all going to be in memory, you might as well start with the very 
large buffer. There is nothing special about Go in this regard - it’s standard 
IO processing. 

> On Dec 29, 2019, at 9:21 AM, Ron Wahler  wrote:
> 
> 
> Jake,
> 
> Thanks for the reply. Csrc.Read is what I was referring to as the connection 
> standard read, should not have used the words "standard read" sorry about 
> that. The problem I am trying to solve is reading an unknown amount of byte 
> data.  I am trying to understand what triggers the Csrc.Read(buf) to return 
> when I send say 3 bytes to it with a client, I also keep the connection open 
> and send a few bytes of characters with the netcat tool, the Csrc.Read 
> returns, but the snip it below that with ReadAll does not return. I am trying 
> to understand the underlying behavior of what triggers a return with the data 
> in these two calls ?
> 
> on this read :
> 
> Csrc net.Conn
> 
> 
>  buf := make([]byte, 1024*32)
> 
>  // READ FROM CLIENT
> 
>  nBytes, err := Csrc.Read(buf)
> 
> 
> 
> Csrc.Read(buf)  returns with a few bytes that I send to it.  It does not wait 
> for the entire allocated buf size to return. This works great, but I am 
> looking for a way to not preallocate a large buffer.
> 
> 
> 
> I am prototyping with ReadAll, see the following snip it, but when I send a 
> few bytes to this call with a client, it does not return. The documentation 
> is saying it may be  looking for an EOF which I do not send.
> 
> 
> 
>   buf, read_err := ioutil.ReadAll(Csrc)
> 
> 
> 
> 
> 
> thanks,
> 
> Ron
> 
> 
> 
> 
> 
> 
> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>> 
>> I am looking for a net.conn standard read that would return a data buffer 
>> the exact size of the read. I am trying to read an unknown amount of byte 
>> data from the connection. With the read i am using I am required to 
>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>> read that works more like  the ReadString , but is for a byte slice.
>> 
>> // I want something similar to this read that returns the read string into 
>> the message string.
>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>> 
>> if ( err != nil ){
>> 
>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>> err)
>> 
>> return 
>> 
>> }
>> 
>> 
>> 
>> 
>> 
>> // had to preallocate a buffer, but I want a read to return me a buffer so I 
>> don't have to guess how big to make it.
>> 
>>  buf := make([]byte, 1024*32)
>> 
>>  // READ FROM CLIENT
>> 
>>  nBytes, err := Csrc.Read(buf)
>> 
>> 
>> 
>> 
>> 
>> Is this not possible, I have not seen any examples that would indicate that 
>> there is a standard library that would do something like what I am looking 
>> for.
>> 
>> 
>> 
>> thanks,
>> 
>> Ron
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/cd1f26d6-72aa-4239-83f2-18dc6220c2db%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/BFCC2BF2-73AA-4539-B833-5CBDF595CC7E%40ix.netcom.com.

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

I agree. I meant that worker pools are especially useful when you can do cpu 
affinity - doesn’t apply to Go. 

I think Go probably needs some idea of “capping” for cpu based workloads. You 
can cap in the local N CPUs , but in a larger app that has multiple parallel 
processing points you can easy create too many routines and then you are paying 
overhead switching costs for no reason. 

> On Dec 28, 2019, at 10:17 AM, Ian Lance Taylor  wrote:
> 
> On Sat, Dec 28, 2019 at 6:11 AM Robert Engels  wrote:
>> 
>> Spinning up a Go routine when for each piece of work may be idiomatic but it 
>> is far from the best practice for many situations - mainly because of cpu 
>> cache invalidation. Most HPC systems create worker pools by type and then 
>> isolate them to cpus/cores - something you can’t do in Go.
>> 
>> I believe there are outstanding proposals for grouping routines and core 
>> affinity.
> 
> I think that today CPU cache invalidation is more or less independent
> of spinning up separate goroutines.  CPU cache invalidation is
> something to consider for threads, but, as you say, goroutines have no
> affinity to threads.  Whether you use a worker pool or not, Go doesn't
> currently give you any way to control how goroutines are assigned to
> threads.  So in general I don't see any reason to think that a worker
> pool would be better or worse with regard to CPU cache invalidation.
> 
> If Go introduces some way to associate goroutines with threads and
> some way to associate threads with CPUs, that might well have an
> effect on whether it is better to use a worker pool.
> 
> Ian
> 
> 
>> On Dec 28, 2019, at 7:02 AM, Agniva De Sarker  
>> wrote:
>> 
>> 
>>> (the task was to limit the whole thing to about 10% of cores)
>> 
>> I still don't think you needed a worker pool here. Like OP mentioned above, 
>> you could just limit the number of goroutines executed to 10% of total cores.
>> 
>> 
>> On Saturday, 28 December 2019 18:02:08 UTC+5:30, Chris Burkert wrote:
>>> 
>>> There are Pros and Cons for everything in life. Some time ago I wrote a 
>>> database tool which does something per table where the runtime largely 
>>> depends on the table size. I started with one goroutine per table because 
>>> it was easy. But that put a high load on the database (the task was to 
>>> limit the whole thing to about 10% of cores) with out of memory situations. 
>>> So I switched to a worker pool which solved that. However now the overall 
>>> runtime was unpredictable. When the large tables were handled at the end 
>>> their runtime defined the overall runtime. So I to feed the pool with large 
>>> tables first. This again lead to out of memory situations so I reordered 
>>> the tables such that large tables are mixed with smaller tables.
>>> 
>>> Brian Candler  schrieb am Sa. 28. Dez. 2019 um 11:09:
>>>> 
>>>> On Friday, 27 December 2019 16:30:48 UTC, Bruno Albuquerque wrote:
>>>>> 
>>>>> This might be useful too you, in any case:
>>>>> 
>>>>> https://git.bug-br.org.br/bga/workerpool
>>>>> 
>>>> 
>>>> I think the point from Bryan Mills' video is, "worker pool" is something 
>>>> of an anti-pattern in go.  goroutines are so cheap that you might as well 
>>>> start a goroutine for each piece of work you have to do, and let it 
>>>> terminate when that piece of work is done.
>>>> 
>>>> Apart from the startup cost, the other reason for having a "worker pool" 
>>>> is to limit the number of concurrent tasks being executed, and there are 
>>>> better ways of doing that in go (also shown in the video).
>>>> 
>>>> --
>>>> You received this message because you are subscribed to the Google Groups 
>>>> "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>>> email to golan...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com.
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/ecddafbc-eb

Re: [go-nuts] Simple worker pool in golnag

[Robert Engels]

Spinning up a Go routine when for each piece of work may be idiomatic but it is 
far from the best practice for many situations - mainly because of cpu cache 
invalidation. Most HPC systems create worker pools by type and then isolate 
them to cpus/cores - something you can’t do in Go.

I believe there are outstanding proposals for grouping routines and core 
affinity. 

> On Dec 28, 2019, at 7:02 AM, Agniva De Sarker  
> wrote:
> 
> 
> > (the task was to limit the whole thing to about 10% of cores)
> 
> I still don't think you needed a worker pool here. Like OP mentioned above, 
> you could just limit the number of goroutines executed to 10% of total cores.
> 
> 
>> On Saturday, 28 December 2019 18:02:08 UTC+5:30, Chris Burkert wrote:
>> There are Pros and Cons for everything in life. Some time ago I wrote a 
>> database tool which does something per table where the runtime largely 
>> depends on the table size. I started with one goroutine per table because it 
>> was easy. But that put a high load on the database (the task was to limit 
>> the whole thing to about 10% of cores) with out of memory situations. So I 
>> switched to a worker pool which solved that. However now the overall runtime 
>> was unpredictable. When the large tables were handled at the end their 
>> runtime defined the overall runtime. So I to feed the pool with large tables 
>> first. This again lead to out of memory situations so I reordered the tables 
>> such that large tables are mixed with smaller tables.
>> 
>> Brian Candler  schrieb am Sa. 28. Dez. 2019 um 11:09:
 On Friday, 27 December 2019 16:30:48 UTC, Bruno Albuquerque wrote:
 This might be useful too you, in any case:
 
 https://git.bug-br.org.br/bga/workerpool
 
>>> 
>>> I think the point from Bryan Mills' video is, "worker pool" is something of 
>>> an anti-pattern in go.  goroutines are so cheap that you might as well 
>>> start a goroutine for each piece of work you have to do, and let it 
>>> terminate when that piece of work is done.
>>> 
>>> Apart from the startup cost, the other reason for having a "worker pool" is 
>>> to limit the number of concurrent tasks being executed, and there are 
>>> better ways of doing that in go (also shown in the video).
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/ecddafbc-eb73-45e1-8a5a-f738e88c6821%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8C1153D5-25E7-4A06-8626-1FE61D9015D5%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

The standard read will early return as soon as some of read is satisfied and a 
subsequent block would occur (or timeout) - so you have to decide when you want 
to stop reading...

>> On Dec 27, 2019, at 9:48 PM, Robert Engels  wrote:
> 
> You need a termination point. In the case of ReadString it is the line 
> terminator. For an arbitrary read it is either a length or EOF - or you can 
> read until the underlying socket has no more data but this is generally 
> useless unless you are doing higher level buffering and protocol parsing. 
> This latter mode is almost never needed in Go due to synchronous nature. 
> 
>>> On Dec 27, 2019, at 9:36 PM, Ron Wahler  wrote:
>> 
>> I did look at ReadAll, but it won't return until it sees EOF.  I am trying 
>> to find something that would return when the standard read would return. I 
>> get the memory part and would manage that. Any other ideas ?
>> 
>> thanks,
>> Ron 
>> 
>>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>>> I am looking for a net.conn standard read that would return a data buffer 
>>> the exact size of the read. I am trying to read an unknown amount of byte 
>>> data from the connection. With the read i am using I am required to 
>>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>>> read that works more like  the ReadString , but is for a byte slice.
>>> 
>>> // I want something similar to this read that returns the read string into 
>>> the message string.
>>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>>> 
>>> if ( err != nil ){
>>> 
>>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>>> err)
>>> 
>>> return 
>>> 
>>> }
>>> 
>>> 
>>> 
>>> 
>>> 
>>> // had to preallocate a buffer, but I want a read to return me a buffer so 
>>> I don't have to guess how big to make it.
>>> 
>>>  buf := make([]byte, 1024*32)
>>> 
>>>  // READ FROM CLIENT
>>> 
>>>  nBytes, err := Csrc.Read(buf)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Is this not possible, I have not seen any examples that would indicate that 
>>> there is a standard library that would do something like what I am looking 
>>> for.
>>> 
>>> 
>>> 
>>> thanks,
>>> 
>>> Ron
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/8ca83343-399d-47ef-aee0-0e173946c980%40googlegroups.com.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/8F20C49C-7240-4E58-A25D-583BD0482115%40ix.netcom.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/26594CE8-822F-44A7-A9FC-D894EE6C8D16%40ix.netcom.com.

Re: [go-nuts] Re: net.conn TCP connection

[Robert Engels]

You need a termination point. In the case of ReadString it is the line 
terminator. For an arbitrary read it is either a length or EOF - or you can 
read until the underlying socket has no more data but this is generally useless 
unless you are doing higher level buffering and protocol parsing. This latter 
mode is almost never needed in Go due to synchronous nature. 

> On Dec 27, 2019, at 9:36 PM, Ron Wahler  wrote:
> 
> 
> I did look at ReadAll, but it won't return until it sees EOF.  I am trying to 
> find something that would return when the standard read would return. I get 
> the memory part and would manage that. Any other ideas ?
> 
> thanks,
> Ron 
> 
>> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>> I am looking for a net.conn standard read that would return a data buffer 
>> the exact size of the read. I am trying to read an unknown amount of byte 
>> data from the connection. With the read i am using I am required to 
>> pre-allocate a buffer and pass that buffer to the read. I am looking for a 
>> read that works more like  the ReadString , but is for a byte slice.
>> 
>> // I want something similar to this read that returns the read string into 
>> the message string.
>>  message, err := bufio.NewReader(ServerConn).ReadString('\n')
>> 
>> if ( err != nil ){
>> 
>> fmt.Println("RELAY: ERROR:  Reg Message read err:", 
>> err)
>> 
>> return 
>> 
>> }
>> 
>> 
>> 
>> 
>> 
>> // had to preallocate a buffer, but I want a read to return me a buffer so I 
>> don't have to guess how big to make it.
>> 
>>  buf := make([]byte, 1024*32)
>> 
>>  // READ FROM CLIENT
>> 
>>  nBytes, err := Csrc.Read(buf)
>> 
>> 
>> 
>> 
>> 
>> Is this not possible, I have not seen any examples that would indicate that 
>> there is a standard library that would do something like what I am looking 
>> for.
>> 
>> 
>> 
>> thanks,
>> 
>> Ron
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/8ca83343-399d-47ef-aee0-0e173946c980%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8F20C49C-7240-4E58-A25D-583BD0482115%40ix.netcom.com.

Re: [go-nuts] Simple worker pool in golnag

[robert engels]

I think it is more complex - or simpler :) - than that. A lot depends on the 
Kafka client - for example the sarama client recommends one client per 
producer/consumer, other clients may multiplex on the same client so having 
more than one consumer (sender) may not be beneficial if the IO is fully async 
acknowledged.

In general, you want to parallelize (add go routines) to the the portions that 
can be parallized (either because they benefit from additional cpu, 
scatter/gather IO (network requests or disk), or have independent/multiple 
destination output stages) - but you have to pay special attention to any 
“ordering” of events that may be required on the consumer side, and the 
acknowledgements required on the Kafka side.

In your example, you are still creating 7xNCPU senders, and only 2 producers - 
which would mean that each send is completely independent and is a minimum 25x 
slower than the producing (given 8 cores x 7 / 2 producers). This could be the 
case, but seems unlikely.

> On Dec 26, 2019, at 11:51 PM, Amarjeet Anand  
> wrote:
> 
> Hi Robert
> 
> Actually the code above is simplified to make it easy to understand.
> 
> Thanks for the suggestion on variable namings... Will improve that.
> 
> The scenario is like the producer functions(produceTaskOfType1ToChan() and 
> produceTaskOfType2ToChan()) will produce a list of strings to the channel... 
> like...
> 
> func produceTaskOfType2ToChan(wg *sync.WaitGroup, autoCancelChan chan string) 
> {
> defer wg.Done()
> autoCancelIds := getAutoCancelIdsFromSource2()
> for autoCancelId := range autoCancelIds {
> autoCancelChan <- autoCancelId
> }
> }
> 
> Now does this code makes some sense?
> 
> 
> 
> On Fri, 27 Dec, 2019, 10:10 AM robert engels,  <mailto:reng...@ix.netcom.com>> wrote:
> Yes, the code doesn’t work :) - it will only ever produce 2 items - unless 
> that was expected - even so, you want the N workers doing work, and probably 
> a constant number sending to Kafka - but a lot depends on your “serial 
> needs”. In your case you only have 2 workers producing work, and N senders - 
> which is backwards to me.
> 
> I would also say that your variable names could be improved - as 
> “autoCancelChan” isn’t really meaningful here, it is simple a chan used to 
> send items to the Kafka senders (at least I think).
> 
>> On Dec 26, 2019, at 10:12 PM, Amarjeet Anand > <mailto:amarjeetanandsi...@gmail.com>> wrote:
>> 
>> Hi
>> 
>> I have to produce some task to kafka parallely. So I want to implement a 
>> simple worker group pattern in go.
>> 
>> Does the below code decent enough to take it to production?
>> 
>> 
>> var workerCount = runtime.NumCPU()*7 + 1
>> 
>> func WorkerPattern() {
>> taskWg := &sync.WaitGroup{}
>> taskWg.Add(2)
>> 
>> autoCancelChan := make(chan string, workerCount*3) // *3, just to make 
>> enough room. workers will be slower anyways
>> go produceTaskOfType1ToChan(taskWg, autoCancelChan)
>> go produceTaskOfType2ToChan(taskWg, autoCancelChan)
>> 
>> // start workers to push autoCancel to kafka
>> workerWg := &sync.WaitGroup{}
>> go kafkaProducerWorkers(autoCancelChan, workerWg)
>> 
>> // wait to close autoCancelChan channel till all the task is written
>> taskWg.Wait()
>> close(autoCancelChan)
>> 
>> // wait till all workers finish their task
>> workerWg.Wait()
>> 
>> fmt.Println("Done!!!")
>> }
>> 
>> func produceTaskOfType1ToChan(wg *sync.WaitGroup, autoCancelChan chan 
>> string) {
>> defer wg.Done()
>> // can produce random number of task on autoCancelChan
>> autoCancelChan <- "task of type of 1"
>> }
>> 
>> func produceTaskOfType2ToChan(wg *sync.WaitGroup, autoCancelChan chan 
>> string) {
>> defer wg.Done()
>> // can produce random number of task on autoCancelChan
>> autoCancelChan <- "task of type of 2"
>> }
>> 
>> func kafkaProducerWorkers(autoCancelChan chan string, workerWg 
>> *sync.WaitGroup) {
>> workerWg.Add(workerCount)
>> for i := 0; i < workerCount; i++ {
>> go produceToKafka(autoCancelChan, workerWg)
>> }
>> }
>> 
>> func produceToKafka(autoCancelChan chan string, workerWg *sync.WaitGroup) {
>> defer workerWg.Done()
>> 
>> // for loop will terminate once autoCancelChan is closed
>> for autoCancel := range autoCancelChan {
>> KafkaClient.PublishToKafkaTopic(autoCancel)
>> }
>> }
>> Any impr

Re: [go-nuts] Simple worker pool in golnag

[robert engels]

Yes, the code doesn’t work :) - it will only ever produce 2 items - unless that 
was expected - even so, you want the N workers doing work, and probably a 
constant number sending to Kafka - but a lot depends on your “serial needs”. In 
your case you only have 2 workers producing work, and N senders - which is 
backwards to me.

I would also say that your variable names could be improved - as 
“autoCancelChan” isn’t really meaningful here, it is simple a chan used to send 
items to the Kafka senders (at least I think).

> On Dec 26, 2019, at 10:12 PM, Amarjeet Anand  
> wrote:
> 
> Hi
> 
> I have to produce some task to kafka parallely. So I want to implement a 
> simple worker group pattern in go.
> 
> Does the below code decent enough to take it to production?
> 
> 
> var workerCount = runtime.NumCPU()*7 + 1
> 
> func WorkerPattern() {
> taskWg := &sync.WaitGroup{}
> taskWg.Add(2)
> 
> autoCancelChan := make(chan string, workerCount*3) // *3, just to make 
> enough room. workers will be slower anyways
> go produceTaskOfType1ToChan(taskWg, autoCancelChan)
> go produceTaskOfType2ToChan(taskWg, autoCancelChan)
> 
> // start workers to push autoCancel to kafka
> workerWg := &sync.WaitGroup{}
> go kafkaProducerWorkers(autoCancelChan, workerWg)
> 
> // wait to close autoCancelChan channel till all the task is written
> taskWg.Wait()
> close(autoCancelChan)
> 
> // wait till all workers finish their task
> workerWg.Wait()
> 
> fmt.Println("Done!!!")
> }
> 
> func produceTaskOfType1ToChan(wg *sync.WaitGroup, autoCancelChan chan string) 
> {
> defer wg.Done()
> // can produce random number of task on autoCancelChan
> autoCancelChan <- "task of type of 1"
> }
> 
> func produceTaskOfType2ToChan(wg *sync.WaitGroup, autoCancelChan chan string) 
> {
> defer wg.Done()
> // can produce random number of task on autoCancelChan
> autoCancelChan <- "task of type of 2"
> }
> 
> func kafkaProducerWorkers(autoCancelChan chan string, workerWg 
> *sync.WaitGroup) {
> workerWg.Add(workerCount)
> for i := 0; i < workerCount; i++ {
> go produceToKafka(autoCancelChan, workerWg)
> }
> }
> 
> func produceToKafka(autoCancelChan chan string, workerWg *sync.WaitGroup) {
> defer workerWg.Done()
> 
> // for loop will terminate once autoCancelChan is closed
> for autoCancel := range autoCancelChan {
> KafkaClient.PublishToKafkaTopic(autoCancel)
> }
> }
> Any improvement you can suggest to this code?
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CANFuhy8qBjooo_tB_gT0f%3DTE4DaOFqWL5SWwNghy%2BL-eV82KdA%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4BD6D8C1-FC0F-4B65-B696-146E3B9E4EFA%40ix.netcom.com.

Re: [go-nuts] Now that wasm is 'standard'... bundle common lang runtimes with browsers

[Robert Engels]

Pretty much the JVM in the browser all over again... if I can live long enough 
maybe I can see people touting the ergonomics and simplicity of the 3270. 

> On Dec 22, 2019, at 5:33 AM, 'Axel Wagner' via golang-nuts 
>  wrote:
> 
> ISTM that this requires being able to distribute the runtime of the language 
> separately from the "user code" and link them together again successfully. 
> Once you have that - it seems a problem far better solved by decent caching. 
> That is, as a Go-user, I'd put a "load Go runtime from 
> https://golang.org/wasm/" or something in my page. That 
> resource is then infinitely cacheable, thus must only be downloaded 
> approximately once (just as if it's bundled). What you get is, that as a 
> developer, you don't need to wait until your favorite version of the runtime 
> of your favorite language is available in all common browsers.
> 
> I'd see "the runtime is bundled with a browser" as replicating exactly the 
> main problem with javascript - that you can't use all its features, because 
> you constantly have to worry about whether or not it's supported in the 
> browser. It seems to subvert what I'd perceive as the main goal of wasm: 
> Decouple the browser engine from the code the developer wants to run in it.
> 
> 
> 
>> On Sun, Dec 22, 2019 at 9:16 AM Russtopia  wrote:
>> Crazy idea, but:
>> 
>> Now that wasm is 'official', why not push browser vendors to put the Go 
>> runtime/stdlib, built for wasm, right into the browser so it doesn't need to 
>> be downloaded with every single *.wasm project?
>> 
>> Of course, to be fair, not just Go -- why not also python-wasm-runtime, 
>> c-wasm-runtime, rust-std-runtime, ...
>> 
>> This would require some sort of 'runtime plugin' standard with versioning, 
>> feature-query and supply-chain security/update mechanisms, of course, but 
>> wouldn't it be nice?
>> 
>> I've been playing with gopherjs and go-wasm, and the big downside to both is 
>> that the baseline *.js file is 1.5-2MB or more, due to bundling a complete 
>> language runtime that must download with every app.
>> 
>> With official 'blessed' wasm runtimes for major languages we could finally 
>> break the deadlock on JS in the browser.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAN4yCu_nHObFToYaM-7CwuiK42EWGinVZNGZt0Ow4x73DXu52Q%40mail.gmail.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGF%2BYbMWk0hAzU2GTFLXLbsJ4pvurejVVNrZF43Yh0kZA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9F77F9D3-27A2-4218-A90D-9A379F626DDD%40ix.netcom.com.

Re: [go-nuts] cgo & long jump

[Robert Engels]

That usage of setjmp and longjmp are pretty much banned in modern C by every 
linter. You can only call setjmp/longjmp within the current call stack (at 
least that's my understanding) - intermixing the context switch to Go would not 
be correct.

Just have a wrapper on f2() that takes the setjmp address() and set it and call 
f2().



-Original Message-
>From: p...@cpan.org
>Sent: Dec 9, 2019 12:30 PM
>To: Ian Lance Taylor 
>Cc: golang-nuts 
>Subject: Re: [go-nuts] cgo & long jump
>
>On Monday 09 December 2019 10:18:26 Ian Lance Taylor wrote:
>> On Mon, Dec 9, 2019 at 8:57 AM  wrote:
>> >
>> > I would like to ask some technical question about cgo and long jumps.
>> >
>> > In Go I have written shared library which exports C API and therefore
>> > can be called by other C applications. This my library is compiling with
>> > cgo, more precisely via: go build -buildmode=c-shared -compiler gc
>> >
>> > Now I would like to call from my Go library some C function which may
>> > call longjmp(). And setjmp() was already called by application which is
>> > using my Go library.
>> >
>> > It is possible to call from Go C function which calls longjmp() which
>> > jumps outside of that my Go library? What would happen with Go garbage
>> > collector and "defer" code?
>> >
>> > To imagine, whole stack looks like this:
>> >
>> >   +--+
>> >   | C application|
>> >   | main():  |
>> >   |   call setjmp()  |
>> >   |   call f1() from Go library  |
>> >   |   ...|
>> >   +--+
>> >   | Go library with exported C f1() function |
>> >   | f1():|
>> >   |   do something in Go |
>> >   |   call f2() from C library   |
>> >   |   ...|
>> >   |   return from f1()   |
>> >   +--+
>> >   | C library with exported f2() function|
>> >   | f2():|
>> >   |   do something in C  |
>> >   |   if error call longjmp()|
>> >   |   else return from f2()  |
>> >   +--+
>> >
>> > And if longjmp() is called then it jumps to main() where setjmp() was
>> > called. So effectively f1() function (in Go) does not return.
>> 
>> I'm pretty sure that won't work at all.  Sorry.
>
>Hi Ian! Thank you for your answer.
>
>I was trying to find any resource on Internet about long jumps and
>"import C" but neither in official Go documentation nor in any other
>blog post is written about it.
>
>There are many resources how Go is (fully) compatible with C and can
>call C functions, but absolutely zero information how it is implemented
>nor what would happen with such core feature of C language as long jumps
>or signals.
>
>> Certainly no Go deferred functions will be run.  I don't think this
>> would break the garbage collector as such, but I expect that it would
>> break the Go scheduler.  In the best case you'll be left with a
>> dangling goroutine that will never be run and never go away.
>
>So should I interpret your answer as long jumps are not supported and
>caller of C function from Go must ensure that called C function would
>not call longjmp()?
>
>
>And another interesting question, it is possible to call longjmp() from
>Go code? To throw C exception back to the main C application.
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to golang-nuts+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/golang-nuts/20191209183039.khrexxnpedzitepy%40pali.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/168169317.2354.1575919493589%40wamui-lola.atl.sa.earthlink.net.

Re: [go-nuts] Help with Go channels and select talk

[Robert Engels]

I’m sorry, but it’s very hard to understand when you start with solutions. I 
think maybe clearly restating the problem will allow more people to offer up 
ideas. To be honest at this point I’m not really certain what you’re trying to 
demonstrate or why. 

> On Dec 8, 2019, at 12:44 AM, Egon Kocjan  wrote:
> 
> I meant lock-free as in "without explicit locks".
> 
> The original challenge still stands if someone has a better solution than me:
> "The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong 
> implementation of bidi-comm, which is what I'll be illustrating. I have three 
> working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can we remove 
> the extra goroutine from 1_1.go and make the code nicer to read than 2_3.go 
> and 2_4.go. The extra goroutine that I'd like to be removed is started here:
> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)"
> 
>> On Sunday, December 8, 2019 at 7:18:16 AM UTC+1, Robert Engels wrote:
>> I understand what you are saying but I’ll still suggest that your 
>> premise/design is not correct. There are plenty of useful lock free 
>> structures in Go (see github.com/robaho/go-concurrency-test) but that is not 
>> what you are attempting here... you are using async processing - these are 
>> completely different things. Using async in Go is an anti-pattern IMO. 
>> 
>>> On Dec 8, 2019, at 12:11 AM, Egon Kocjan  wrote:
>>> 
>>> 
>>> I'll cite myself:
>>> "I'm preparing a short talk about Go channels and select. More 
>>> specifically, I want to show what not to do."
>>> and
>>> "it would be tempting to just combine two goroutines into one and handle 
>>> caching in a single loop without using locks (I see developers avoid 
>>> atomics and locks if they don't have a lot of previous experience with 
>>> traditional MT primitives)"
>>> 
>>> Before I say one can't do something in Go, I wanted to ask here to make 
>>> sure I'm not missing something obvious. Basically, I intend to show how 
>>> difficult lock-free programming can be so don't force it - just use 
>>> goroutines and locks.
>>> 
>>>> On Saturday, December 7, 2019 at 3:46:43 PM UTC+1, Robert Engels wrote:
>>>> Probably not. Go is designed for 1:1 and there is no reason to do it 
>>>> differently. You could probably try to write an async event driven layer 
>>>> (which it looks like you’ve tried) but why???
>>>> 
>>>> It’s like saying I’d really like my plane to float - you can do that -but 
>>>> most likely you want a boat instead of a plane. 
>>>> 
>>>>> On Dec 7, 2019, at 2:38 AM, Egon Kocjan  wrote:
>>>>> 
>>>>> 
>>>>> I'll try to clarify as best as I can, thanks again to anyone looking at 
>>>>> this.
>>>>> 
>>>>> The simple server implementation of "output <- input+1" is here and it is 
>>>>> not "under our control" - it's what we have to work with: 
>>>>> https://github.com/egonk/chandemo/blob/master/server.go
>>>>> 
>>>>> The test runner or client is here: 
>>>>> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in 
>>>>> ints and gets server replies back through a connection layer)
>>>>> 
>>>>> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong 
>>>>> implementation of bidi-comm, which is what I'll be illustrating. I have 
>>>>> three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can 
>>>>> we remove the extra goroutine from 1_1.go and make the code nicer to read 
>>>>> than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed 
>>>>> is started here:
>>>>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)
>>>>> 
>>>>> What I mean by removed - no go statement, replaced presumably by some 
>>>>> kind of for/select combination.
>>>>> 
>>>>>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote:
>>>>>> I’m sorry but your design is not comprehendible by me, and I’ve done 
>>>>>> lots of TCP based services. 
>>>>>> 
>>>>>> i think you only need to emulate classic TCP processing - a reader 
>>>>>> thread (Go routine) on each side of the 

Re: [go-nuts] Help with Go channels and select talk

[Robert Engels]

I understand what you are saying but I’ll still suggest that your 
premise/design is not correct. There are plenty of useful lock free structures 
in Go (see github.com/robaho/go-concurrency-test) but that is not what you are 
attempting here... you are using async processing - these are completely 
different things. Using async in Go is an anti-pattern IMO. 

> On Dec 8, 2019, at 12:11 AM, Egon Kocjan  wrote:
> 
> 
> I'll cite myself:
> "I'm preparing a short talk about Go channels and select. More specifically, 
> I want to show what not to do."
> and
> "it would be tempting to just combine two goroutines into one and handle 
> caching in a single loop without using locks (I see developers avoid atomics 
> and locks if they don't have a lot of previous experience with traditional MT 
> primitives)"
> 
> Before I say one can't do something in Go, I wanted to ask here to make sure 
> I'm not missing something obvious. Basically, I intend to show how difficult 
> lock-free programming can be so don't force it - just use goroutines and 
> locks.
> 
>> On Saturday, December 7, 2019 at 3:46:43 PM UTC+1, Robert Engels wrote:
>> Probably not. Go is designed for 1:1 and there is no reason to do it 
>> differently. You could probably try to write an async event driven layer 
>> (which it looks like you’ve tried) but why???
>> 
>> It’s like saying I’d really like my plane to float - you can do that -but 
>> most likely you want a boat instead of a plane. 
>> 
>>>> On Dec 7, 2019, at 2:38 AM, Egon Kocjan  wrote:
>>>> 
>>> 
>>> I'll try to clarify as best as I can, thanks again to anyone looking at 
>>> this.
>>> 
>>> The simple server implementation of "output <- input+1" is here and it is 
>>> not "under our control" - it's what we have to work with: 
>>> https://github.com/egonk/chandemo/blob/master/server.go
>>> 
>>> The test runner or client is here: 
>>> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in 
>>> ints and gets server replies back through a connection layer)
>>> 
>>> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong 
>>> implementation of bidi-comm, which is what I'll be illustrating. I have 
>>> three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can 
>>> we remove the extra goroutine from 1_1.go and make the code nicer to read 
>>> than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed is 
>>> started here:
>>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)
>>> 
>>> What I mean by removed - no go statement, replaced presumably by some kind 
>>> of for/select combination.
>>> 
>>>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote:
>>>> I’m sorry but your design is not comprehendible by me, and I’ve done lots 
>>>> of TCP based services. 
>>>> 
>>>> i think you only need to emulate classic TCP processing - a reader thread 
>>>> (Go routine) on each side of the connection using range to read until 
>>>> closed. The connection is represented by 2 channels - one for each 
>>>> direction.
>>>> 
>>>> I think you might be encountering a deadlock because the producer on one 
>>>> end is not also reading the incoming - so either restructure, or use 2 
>>>> more threads for the producers.
>>>> 
>>>> 
>>>> 
>>>>> On Dec 6, 2019, at 10:38 PM, Egon Kocjan  wrote:
>>>>> 
>>>>> Agreed, I see goroutines in general as a big win. But what I intend to 
>>>>> talk about in the presentation:
>>>>> - we have two unidirectional flows of data resembling something like a 
>>>>> TCP socket, easy to do with two goroutines with a for loop
>>>>> - let's add caching, so some requests do not go to the server
>>>>> - it would be tempting to just combine two goroutines into one and handle 
>>>>> caching in a single loop without using locks (I see developers avoid 
>>>>> atomics and locks if they don't have a lot of previous experience with 
>>>>> traditional MT primitives)
>>>>> - this is surprisingly difficult to do properly with Go channels, see my 
>>>>> attempts: https://github.com/egonk/chandemo/blob/master/2_3.go and 
>>>>> https://github.com/egonk/chandemo/blob/master/2_4.go
>>>>

Re: [go-nuts] Help with Go channels and select talk

[Robert Engels]

Probably not. Go is designed for 1:1 and there is no reason to do it 
differently. You could probably try to write an async event driven layer (which 
it looks like you’ve tried) but why???

It’s like saying I’d really like my plane to float - you can do that -but most 
likely you want a boat instead of a plane. 

> On Dec 7, 2019, at 2:38 AM, Egon Kocjan  wrote:
> 
> 
> I'll try to clarify as best as I can, thanks again to anyone looking at this.
> 
> The simple server implementation of "output <- input+1" is here and it is not 
> "under our control" - it's what we have to work with: 
> https://github.com/egonk/chandemo/blob/master/server.go
> 
> The test runner or client is here: 
> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in ints 
> and gets server replies back through a connection layer)
> 
> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong 
> implementation of bidi-comm, which is what I'll be illustrating. I have three 
> working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can we remove 
> the extra goroutine from 1_1.go and make the code nicer to read than 2_3.go 
> and 2_4.go. The extra goroutine that I'd like to be removed is started here:
> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)
> 
> What I mean by removed - no go statement, replaced presumably by some kind of 
> for/select combination.
> 
>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote:
>> I’m sorry but your design is not comprehendible by me, and I’ve done lots of 
>> TCP based services. 
>> 
>> i think you only need to emulate classic TCP processing - a reader thread 
>> (Go routine) on each side of the connection using range to read until 
>> closed. The connection is represented by 2 channels - one for each direction.
>> 
>> I think you might be encountering a deadlock because the producer on one end 
>> is not also reading the incoming - so either restructure, or use 2 more 
>> threads for the producers.
>> 
>> 
>> 
>>> On Dec 6, 2019, at 10:38 PM, Egon Kocjan  wrote:
>>> 
>>> Agreed, I see goroutines in general as a big win. But what I intend to talk 
>>> about in the presentation:
>>> - we have two unidirectional flows of data resembling something like a TCP 
>>> socket, easy to do with two goroutines with a for loop
>>> - let's add caching, so some requests do not go to the server
>>> - it would be tempting to just combine two goroutines into one and handle 
>>> caching in a single loop without using locks (I see developers avoid 
>>> atomics and locks if they don't have a lot of previous experience with 
>>> traditional MT primitives)
>>> - this is surprisingly difficult to do properly with Go channels, see my 
>>> attempts: https://github.com/egonk/chandemo/blob/master/2_3.go and 
>>> https://github.com/egonk/chandemo/blob/master/2_4.go
>>> - it is easy to do in actor systems, just move the code for both actors 
>>> into a single actor!
>>> 
>>> The lesson here is that select is not a nice and safe compose statement 
>>> even if it appears so at the first glance, do not be afraid to use locks.
>>> 
>>> Of course, if somebody comes up with a better implementation than 2_3.go 
>>> and 2_4.go, I would be very happy to include it in the talk.
>>> 
>>>> On Saturday, December 7, 2019 at 4:17:04 AM UTC+1, robert engels wrote:
>>>> To clarify, with Go’s very lightweight threads it is “doing the 
>>>> multiplexing for you” - often only a single CPU is consumed if the 
>>>> producer and consumer work cannot be parallelized, otherwise you get this 
>>>> concurrency “for free”.
>>>> 
>>>> You are trying to manually perform the multiplexing - you need async 
>>>> structures to do this well - Go doesn’t really support async by design - 
>>>> and it’s a much simpler programming model as a result.
>>>> 
>>>>> On Dec 6, 2019, at 12:02 PM, Robert Engels  wrote:
>>>>> 
>>>>> A channel is much closer to a pipe. There are producers and consumers and 
>>>>> these are typically different threads of execution unless you have an 
>>>>> event based (async) system - that is not Go. 
>>>>> 
>>>>>> On Dec 6, 2019, at 9:30 AM, Egon Kocjan  wrote:
>>>>>> 
>>>>>> 
>>>>>> There are goroutines in the examples of course, just a single goroutine 
>>>>>>

Re: [go-nuts] Help with Go channels and select talk

[robert engels]

I’m sorry but your design is not comprehendible by me, and I’ve done lots of 
TCP based services. 

i think you only need to emulate classic TCP processing - a reader thread (Go 
routine) on each side of the connection using range to read until closed. The 
connection is represented by 2 channels - one for each direction.

I think you might be encountering a deadlock because the producer on one end is 
not also reading the incoming - so either restructure, or use 2 more threads 
for the producers.



> On Dec 6, 2019, at 10:38 PM, Egon Kocjan  wrote:
> 
> Agreed, I see goroutines in general as a big win. But what I intend to talk 
> about in the presentation:
> - we have two unidirectional flows of data resembling something like a TCP 
> socket, easy to do with two goroutines with a for loop
> - let's add caching, so some requests do not go to the server
> - it would be tempting to just combine two goroutines into one and handle 
> caching in a single loop without using locks (I see developers avoid atomics 
> and locks if they don't have a lot of previous experience with traditional MT 
> primitives)
> - this is surprisingly difficult to do properly with Go channels, see my 
> attempts: https://github.com/egonk/chandemo/blob/master/2_3.go 
> <https://github.com/egonk/chandemo/blob/master/2_3.go> and 
> https://github.com/egonk/chandemo/blob/master/2_4.go 
> <https://github.com/egonk/chandemo/blob/master/2_3.go>
> - it is easy to do in actor systems, just move the code for both actors into 
> a single actor!
> 
> The lesson here is that select is not a nice and safe compose statement even 
> if it appears so at the first glance, do not be afraid to use locks.
> 
> Of course, if somebody comes up with a better implementation than 2_3.go and 
> 2_4.go, I would be very happy to include it in the talk.
> 
> On Saturday, December 7, 2019 at 4:17:04 AM UTC+1, robert engels wrote:
> To clarify, with Go’s very lightweight threads it is “doing the multiplexing 
> for you” - often only a single CPU is consumed if the producer and consumer 
> work cannot be parallelized, otherwise you get this concurrency “for free”.
> 
> You are trying to manually perform the multiplexing - you need async 
> structures to do this well - Go doesn’t really support async by design - and 
> it’s a much simpler programming model as a result.
> 
>> On Dec 6, 2019, at 12:02 PM, Robert Engels > wrote:
>> 
>> A channel is much closer to a pipe. There are producers and consumers and 
>> these are typically different threads of execution unless you have an event 
>> based (async) system - that is not Go. 
>> 
>>> On Dec 6, 2019, at 9:30 AM, Egon Kocjan > wrote:
>>> 
>>> 
>>> There are goroutines in the examples of course, just a single goroutine per 
>>> bidi channel seems hard. By contrast, I've worked with actor systems before 
>>> and they are perfectly fine with a single fiber.
>>> 
>>> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels wrote:
>>> Channels are designed to be used with multiple go routines - if you’re not 
>>> you are doing something wrong. 
>>> 
>>>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan > wrote:
>>>> 
>>>> 
>>>> Hello
>>>> 
>>>> I'm preparing a short talk about Go channels and select. More 
>>>> specifically, I want to show what not to do. I chose a bidirectional 
>>>> communication channel implementation, because it seems to be a common base 
>>>> for a lot of problems but hard to implement correctly without using any 
>>>> extra goroutines. All the code is here: https://github.com/egonk/chandemo 
>>>> <https://github.com/egonk/chandemo>
>>>> 
>>>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints)
>>>> 2_1.go: nice but completely wrong
>>>> 2_2.go: better but still deadlocks
>>>> 2_3.go: correct but ugly and slow (takes more than 2s for million ints)
>>>> 2_4.go: correct and a bit faster but still ugly (1.8s for million ints)
>>>> 
>>>> So my question: is there a better way of doing it with just nested for and 
>>>> select and no goroutines? Basically, what would 2_5.go look like?
>>>> 
>>>> Thank you
>>>> Egon
>>>> 
>>>> -- 
>>>> You received this message because you are subscribed to the Google Groups 
>>>> "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>>> email to golan...@googlegroups.com <>

Re: [go-nuts] Help with Go channels and select talk

[robert engels]

To clarify, with Go’s very lightweight threads it is “doing the multiplexing 
for you” - often only a single CPU is consumed if the producer and consumer 
work cannot be parallelized, otherwise you get this concurrency “for free”.

You are trying to manually perform the multiplexing - you need async structures 
to do this well - Go doesn’t really support async by design - and it’s a much 
simpler programming model as a result.

> On Dec 6, 2019, at 12:02 PM, Robert Engels  wrote:
> 
> A channel is much closer to a pipe. There are producers and consumers and 
> these are typically different threads of execution unless you have an event 
> based (async) system - that is not Go. 
> 
>> On Dec 6, 2019, at 9:30 AM, Egon Kocjan  wrote:
>> 
>> 
>> There are goroutines in the examples of course, just a single goroutine per 
>> bidi channel seems hard. By contrast, I've worked with actor systems before 
>> and they are perfectly fine with a single fiber.
>> 
>> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels wrote:
>> Channels are designed to be used with multiple go routines - if you’re not 
>> you are doing something wrong. 
>> 
>>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan > wrote:
>>> 
>>> 
>>> Hello
>>> 
>>> I'm preparing a short talk about Go channels and select. More specifically, 
>>> I want to show what not to do. I chose a bidirectional communication 
>>> channel implementation, because it seems to be a common base for a lot of 
>>> problems but hard to implement correctly without using any extra 
>>> goroutines. All the code is here: https://github.com/egonk/chandemo 
>>> <https://github.com/egonk/chandemo>
>>> 
>>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints)
>>> 2_1.go: nice but completely wrong
>>> 2_2.go: better but still deadlocks
>>> 2_3.go: correct but ugly and slow (takes more than 2s for million ints)
>>> 2_4.go: correct and a bit faster but still ugly (1.8s for million ints)
>>> 
>>> So my question: is there a better way of doing it with just nested for and 
>>> select and no goroutines? Basically, what would 2_5.go look like?
>>> 
>>> Thank you
>>> Egon
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com <>.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com 
>> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F798CDF8-8108-437F-A435-7C8B882BFA96%40ix.netcom.com.

Re: [go-nuts] Help with Go channels and select talk

[Robert Engels]

A channel is much closer to a pipe. There are producers and consumers and these 
are typically different threads of execution unless you have an event based 
(async) system - that is not Go. 

> On Dec 6, 2019, at 9:30 AM, Egon Kocjan  wrote:
> 
> 
> There are goroutines in the examples of course, just a single goroutine per 
> bidi channel seems hard. By contrast, I've worked with actor systems before 
> and they are perfectly fine with a single fiber.
> 
>> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels wrote:
>> Channels are designed to be used with multiple go routines - if you’re not 
>> you are doing something wrong. 
>> 
>>>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan  wrote:
>>>> 
>>> 
>>> Hello
>>> 
>>> I'm preparing a short talk about Go channels and select. More specifically, 
>>> I want to show what not to do. I chose a bidirectional communication 
>>> channel implementation, because it seems to be a common base for a lot of 
>>> problems but hard to implement correctly without using any extra 
>>> goroutines. All the code is here: https://github.com/egonk/chandemo
>>> 
>>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints)
>>> 2_1.go: nice but completely wrong
>>> 2_2.go: better but still deadlocks
>>> 2_3.go: correct but ugly and slow (takes more than 2s for million ints)
>>> 2_4.go: correct and a bit faster but still ugly (1.8s for million ints)
>>> 
>>> So my question: is there a better way of doing it with just nested for and 
>>> select and no goroutines? Basically, what would 2_5.go look like?
>>> 
>>> Thank you
>>> Egon
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9F3C3A18-9A8D-44A6-881F-729FB364FBED%40ix.netcom.com.

Re: [go-nuts] Inconsistent rounding with float printf ?

[Robert Engels]

You can use github.com/robaho/fixed

> On Dec 6, 2019, at 10:19 AM, Michael Jones  wrote:
> 
> 
> Agree with Ian. 
> 
> Solutions are: change expectations, use decimal floating point, or use a 
> base-independent decimal representation. The latter implies scaled integers.
> 
> Quick, ugly, and typed on one hand from bed, but here it is: 
> https://play.golang.org/p/fBztRY6qHP0
> 
> 999000/1000 = 999.0
> 999050/1000 = 999.1
> 999100/1000 = 999.1
> 999150/1000 = 999.2
> 999200/1000 = 999.2
> 999250/1000 = 999.3
> 999300/1000 = 999.3
> 999350/1000 = 999.4
> 999400/1000 = 999.4
> 999450/1000 = 999.5
> 999500/1000 = 999.5
> 999550/1000 = 999.6
> 999600/1000 = 999.6
> 999650/1000 = 999.7
> 999700/1000 = 999.7
> 999750/1000 = 999.8
> 999800/1000 = 999.8
> 999850/1000 = 999.9
> 00/1000 = 999.9
> 50/1000 = 1000.0
> -50/1000 = -1000.0
> -00/1000 = -999.9
> -999850/1000 = -999.9
> -999800/1000 = -999.8
> -999750/1000 = -999.8
> -999700/1000 = -999.7
> -999650/1000 = -999.7
> -999600/1000 = -999.6
> -999550/1000 = -999.6
> -999500/1000 = -999.5
> -999450/1000 = -999.5
> -999400/1000 = -999.4
> -999350/1000 = -999.4
> -999300/1000 = -999.3
> -999250/1000 = -999.3
> -999200/1000 = -999.2
> -999150/1000 = -999.2
> -999100/1000 = -999.1
> -999050/1000 = -999.1
> 
>> On Fri, Dec 6, 2019 at 2:31 AM Ian Davis  wrote:
>>> On Fri, 6 Dec 2019, at 9:25 AM, Christophe Meessen wrote:
>>> I have noticed that printf performs an apparently inconsistent rounding of 
>>> floating point values.
>>> 
>>> I divide a big number by 1000 and printf the resulting value with "%.1f".
>>> Here is the code: https://play.golang.org/p/e7dD3c6IHq2
>> 
>> I think you are just seeing the usual problems of floating point 
>> representation. 
>> 
>> You may wonder why 999450/1000=999.45, 999500/1000=999.50 and 
>> 999550/1000=999.55 all format as 999.5. The answer is that the internal 
>> representation of the three results cannot correspond to the mathematical 
>> result you expect.
>> 
>> This link shows the internal representation of each answer: 
>> https://play.golang.org/p/bBTNCdsAttR
>> 
>> You can see that 999550/1000 = 
>> 999.5499954525264911353588104248046875 which is printed as 999.5
>> 
>> 
>>> I would expect the rounding rule to be "round away from zero" as defined 
>>> here: https://math.stackexchange.com/a/2252888/33796
>>> In this case 0.5 is rounded to 1 (or 0.05 to 0.1) and -0.5 to -1 (or -0.05 
>>> to -0.1). 
>> 
>> The strconv and fmt packages use round to even as a rule. Use math.Round to 
>> round away from zero.
>> 
>> -- Ian
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/ee94624c-5485-4daf-98ad-8e59055056dd%40www.fastmail.com.
> 
> 
> -- 
> Michael T. Jones
> michael.jo...@gmail.com
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CALoEmQy%3DMCWgb69GgsZwAv2GT4ar%3DWrdvn212sFK1PfGES1ijw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/D9F522AC-A89C-4915-9DEE-41D70E8A7C65%40ix.netcom.com.

Re: [go-nuts] Help with Go channels and select talk

[Robert Engels]

Channels are designed to be used with multiple go routines - if you’re not you 
are doing something wrong. 

> On Dec 6, 2019, at 8:32 AM, Egon Kocjan  wrote:
> 
> 
> Hello
> 
> I'm preparing a short talk about Go channels and select. More specifically, I 
> want to show what not to do. I chose a bidirectional communication channel 
> implementation, because it seems to be a common base for a lot of problems 
> but hard to implement correctly without using any extra goroutines. All the 
> code is here: https://github.com/egonk/chandemo
> 
> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints)
> 2_1.go: nice but completely wrong
> 2_2.go: better but still deadlocks
> 2_3.go: correct but ugly and slow (takes more than 2s for million ints)
> 2_4.go: correct and a bit faster but still ugly (1.8s for million ints)
> 
> So my question: is there a better way of doing it with just nested for and 
> select and no goroutines? Basically, what would 2_5.go look like?
> 
> Thank you
> Egon
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/BD2088DD-59DA-4AA4-AE34-C8E81A79982A%40ix.netcom.com.

Re: [go-nuts] log may block the entire program if stderr pipe gets full

[Robert Engels]

Nothing to be done here. This is the expected behavior and has been that way 
since the start of Unix. 

You can get around this by piping to an intermediate process that spools to 
disk and reads from that - until you run out of disk space. 

Honestly you probably should fix the consumer. 

> On Dec 5, 2019, at 11:27 AM, peter.juhas...@gmail.com wrote:
> 
> 
> I've run into an insidious issue.
> 
> The log package writes to the standard error. However, at least on linux, if 
> stderr is redirected to a pipe and the other end of that pipe is not drained, 
> then the kernel buffer associated with that pipe eventually gets full, and 
> once that happens, subsequent write calls to stderr will block indefinitely. 
> The problem is exacerbated by the fact that the log package uses a mutex to 
> ensure that messages don't get mixed up on output - but because the of the 
> blocked write call the mutex is never unlocked, so any subsequent call to any 
> output function of the log package, from anywhere in the program, will block.
> 
> Demonstration: the following program, when ran in itself, prints several 
> dummy strings to stderr then exits with a panic message.
> 
> //
> package main
> 
> import (
> "log"
> "time"
> )
> 
> func spam(id int) {
> max := 1
> for i := 0; i < max; i++ {
> log.Printf("%d\t%06d\t%10s", id, i, "")
> }
> }
> 
> func main() {
> go spam(1)
> go spam(2)
> time.Sleep(1 * time.Second)
> panic("exiting")
> }
> //
> 
> However, when ran through the following script, which redirects the standard 
> error of its child process but purposely doesn't read from it, the program 
> hangs.
> 
> 
> #!/usr/bin/perl
> 
> use strict;
> use warnings;
> use IPC::Open3;
> use Symbol 'gensym';
> 
> my($wtr, $rdr, $err);
> $err = gensym;
> my $pid = open3($wtr, $rdr, $err, @ARGV);
> 
> waitpid( $pid, 0 );
> ##
> 
> (Compile the above go program as `child`, save this script as runner.pl, then 
> run it as `perl runner.pl ./child`.)
> If you run it with strace, you can see that it can't even panic, because it 
> blocks on one of the many write calls that produce the panic message.
> 
> 
> The question of "but why would you do that" may arise - of course I would not 
> *want* to do this. The issue came up in an environment where a go program 
> somewhat more complex than the one above is run by supervisord, and I use 
> supervisord's output redirection facility to forward the go program's stderr 
> to a remote syslog sink. And, for some reason I don't yet understand, 
> supervisord stopped reading the stderr pipe mid-run.
> So I don't say that this is a bug in go, I'm not sure that you could even do 
> anything about it, I just find the issue particularly annoying, because it's 
> hard to detect and hard to work around.
> Perhaps a note in the log package's documentation about possible dangers of 
> redirecting stderr may be useful.
> 
> best regards,
> Peter Juhasz
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/83a396b3-41c1-453d-b604-3640ba58bdb0%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8CABDBE5-11C5-48EC-A4EA-864C3B924B70%40ix.netcom.com.

Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

[Robert Engels]

For completeness, RWMutex.TryLock() is also needed. If you are modifying the stdlib, these are trivial changes to add these methods.-Original Message-
From: Liam 
Sent: Dec 4, 2019 12:34 PM
To: golang-nuts 
Subject: Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

Thank you for the refs and pseudocode!Somehow I imagined this would be simpler... A proposal for Mutex.TryLock() was turned aside years ago because it's trivial to implement with CAS. But they didn't consider RWMutex.TryRLock().https://github.com/golang/go/issues/6123@ianlancetaylor, is this worth a new proposal?On Tuesday, December 3, 2019 at 7:52:54 PM UTC-8, robert engels wrote:You actually only need a cas and a condition variable (technically the condition in Go requires a backing lock, but if you have wait/notify it isn’t needed).You can read this https://code.woboq.org/userspace/glibc/nptl/pthread_rwlock_common.c.html and/or https://6826.csail.mit.edu/2019/papers/HO17.pdf for the general idea.Essentially, you use some bits of the ‘atomic value’ to encode phases (waiting for write lock, holding write lock, reader bits) - and use the cond variable to block the writers (or readers for that matter) and loop and retry the cas.The following is simplified since it isn’t “fair”, but you can add another bit for “writer waiting” to accomplish this. Also, the ‘condition’ must be a ‘flag’ so that a signal() with no waiters is satisfied by the next wait()bits 0 - 30 number of readersbits 31 writer has lockWRITER = 1<<31read() is the atomic read of vpseudo codetry_lock() {  for {	  v = read()  	  if v has writer {              return WOULD_BLOCK          }          if(cas(v,v + 1 reader)) {              return OK          } else {             // loop and try again         }  }}runlock() {   for{      v = read()      if(cas(v,v -1 reader)) {         cond.signal() // can be optimized to not always do this, i.e. only when readers is 0         return      } else {         // loop and try again      }  }}wlock() {   for {      v = read()      if v !=0 {          cond.wait()     }     else {        if(cas(v,WRITER)) {  // we are the writer            return        } else {            // loop and try again        }    }}wunlock() {    set(v,0);    cond.signal() // for other waiting writers}Obviously if you need readers to block on writers (i.e. rlock() ) it is slightly more work, but you get the idea.On Dec 3, 2019, at 7:54 PM, Liam <networ...@gmail.com> wrote:Busy means would-block, yes.Burak thanks, but that doesn't work for read-lock.On Tuesday, December 3, 2019 at 5:39:48 PM UTC-8, Robert Engels wrote:It depends then, because technically the Go RW lock queues readers behind a waiting writer so “busy” is somewhat undefined. If “busy” means “would block” you can still do it - I’ll post the code tonight. 

> On Dec 3, 2019, at 6:49 PM, Robert Engels <ren...@ix.netcom.com> wrote:
> 
> I would use an atomic and a lock instead of two locks. 
> 
>>> On Dec 3, 2019, at 6:35 PM, burak serdar <bse...@computer.org> wrote:
>>> 
>>> On Tue, Dec 3, 2019 at 5:21 PM Liam Breck <networ...@gmail.com> wrote:
>>> 
>>> I have a problem that is trivially solved via
>>> 
>>> door sync.RWMutex
>>> 
>>> func Reader() T {
>>>  if !door.TryRLock() { // missing in stdlib :-(
>>>     return busy
>>>  }
>>>  defer door.RUnlock()
>>>  ...
>>> }
>>> 
>>> func Writer() {
>>>  door.Lock()
>>>  defer door.Unlock()
>>>  ...
>>> }
>>> 
>>> How does one achieve this in Go?
>> 
>> Two locks and a bool?
>> 
>> var door=sync.Mutex{}
>> var x=sync.Mutex{}
>> var b bool
>> 
>> func trylock() bool {
>> x.Lock()
>> if b {
>> x.Unlock()
>> return false
>> }
>> b=true
>> door.Lock()
>> x.Unlock()
>> return true
>> }
>> 
>> unlock:
>> 
>> x.Lock()
>> b=false
>> door.Unlock()
>> x.Unlock()
>> 
>> 
>> 
>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
>> To view this discussion on the web visit https://groups.go

Re: [go-nuts] Re: Workaround for missing RWMutex.Try*Lock()

[Robert Engels]

That pkg is not a RW lock though - it is an exclusive mutex. The pseudo code I 
provided is a RW try lock.  

> On Dec 4, 2019, at 4:49 AM, pierre.cu...@gmail.com wrote:
> 
> 
> This may be of relevance to you:
> https://pkg.go.dev/gvisor.dev/gvisor/pkg/tmutex?tab=doc
> 
> 
> Le mercredi 4 décembre 2019 01:21:41 UTC+1, Liam a écrit :
>> 
>> I have a problem that is trivially solved via
>> 
>> door sync.RWMutex
>> 
>> func Reader() T {
>>if !door.TryRLock() { // missing in stdlib :-(
>>   return busy
>>}
>>defer door.RUnlock()
>>...
>> }
>> 
>> func Writer() {
>>door.Lock()
>>defer door.Unlock()
>>...
>> }
>> 
>> How does one achieve this in Go?
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/94c7a670-0a71-432a-aa19-035e1f4c9648%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/A958A582-0660-4C45-BFB7-7807D052447B%40ix.netcom.com.

Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

[robert engels]

You actually only need a cas and a condition variable (technically the 
condition in Go requires a backing lock, but if you have wait/notify it isn’t 
needed).

You can read this 
https://code.woboq.org/userspace/glibc/nptl/pthread_rwlock_common.c.html 
<https://code.woboq.org/userspace/glibc/nptl/pthread_rwlock_common.c.html> 
and/or https://6826.csail.mit.edu/2019/papers/HO17.pdf 
<https://6826.csail.mit.edu/2019/papers/HO17.pdf> for the general idea.

Essentially, you use some bits of the ‘atomic value’ to encode phases (waiting 
for write lock, holding write lock, reader bits) - and use the cond variable to 
block the writers (or readers for that matter) and loop and retry the cas.

The following is simplified since it isn’t “fair”, but you can add another bit 
for “writer waiting” to accomplish this. Also, the ‘condition’ must be a ‘flag’ 
so that a signal() with no waiters is satisfied by the next wait()

bits 0 - 30 number of readers
bits 31 writer has lock

WRITER = 1<<31
read() is the atomic read of v

pseudo code

try_lock() {
  for {
  v = read()
  if v has writer {
  return WOULD_BLOCK
  }
  if(cas(v,v + 1 reader)) {
  return OK
  } else {
 // loop and try again
 }
  }
}

runlock() {
   for{
  v = read()
  if(cas(v,v -1 reader)) {
 cond.signal() // can be optimized to not always do this, i.e. only 
when readers is 0
 return
  } else {
 // loop and try again
  }
  }
}

wlock() {
   for {
  v = read()
  if v !=0 { 
 cond.wait()
 }
 else {
if(cas(v,WRITER)) {  // we are the writer
return
} else {
// loop and try again
}
}
}

wunlock() {
set(v,0);
cond.signal() // for other waiting writers
}

Obviously if you need readers to block on writers (i.e. rlock() ) it is 
slightly more work, but you get the idea.


> On Dec 3, 2019, at 7:54 PM, Liam  wrote:
> 
> Busy means would-block, yes.
> 
> Burak thanks, but that doesn't work for read-lock.
> 
> On Tuesday, December 3, 2019 at 5:39:48 PM UTC-8, Robert Engels wrote:
> It depends then, because technically the Go RW lock queues readers behind a 
> waiting writer so “busy” is somewhat undefined. If “busy” means “would block” 
> you can still do it - I’ll post the code tonight. 
> 
> > On Dec 3, 2019, at 6:49 PM, Robert Engels > wrote: 
> > 
> > I would use an atomic and a lock instead of two locks. 
> > 
> >>> On Dec 3, 2019, at 6:35 PM, burak serdar > wrote: 
> >>> 
> >>> On Tue, Dec 3, 2019 at 5:21 PM Liam Breck > 
> >>> wrote: 
> >>> 
> >>> I have a problem that is trivially solved via 
> >>> 
> >>> door sync.RWMutex 
> >>> 
> >>> func Reader() T { 
> >>>  if !door.TryRLock() { // missing in stdlib :-( 
> >>> return busy 
> >>>  } 
> >>>  defer door.RUnlock() 
> >>>  ... 
> >>> } 
> >>> 
> >>> func Writer() { 
> >>>  door.Lock() 
> >>>  defer door.Unlock() 
> >>>  ... 
> >>> } 
> >>> 
> >>> How does one achieve this in Go? 
> >> 
> >> Two locks and a bool? 
> >> 
> >> var door=sync.Mutex{} 
> >> var x=sync.Mutex{} 
> >> var b bool 
> >> 
> >> func trylock() bool { 
> >> x.Lock() 
> >> if b { 
> >> x.Unlock() 
> >> return false 
> >> } 
> >> b=true 
> >> door.Lock() 
> >> x.Unlock() 
> >> return true 
> >> } 
> >> 
> >> unlock: 
> >> 
> >> x.Lock() 
> >> b=false 
> >> door.Unlock() 
> >> x.Unlock() 
> >> 
> >> 
> >> 
> >> 
> >>> 
> >>> -- 
> >>> You received this message because you are subscribed to the Google Groups 
> >>> "golang-nuts" group. 
> >>> To unsubscribe from this group and stop receiving emails from it, send an 
> >>> email to golan...@googlegroups.com <>. 
> >>> To view this discussion on the web visit 
> >>> https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com
> >>>  
> >>> <https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com>.
> >>>  
> >> 
> >> -- 
> >> You received this message because you are subscribed to the Google Groups 
> >> "golang-nuts" group. 
> >> To unsub

Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

[Robert Engels]

It depends then, because technically the Go RW lock queues readers behind a 
waiting writer so “busy” is somewhat undefined. If “busy” means “would block” 
you can still do it - I’ll post the code tonight. 

> On Dec 3, 2019, at 6:49 PM, Robert Engels  wrote:
> 
> I would use an atomic and a lock instead of two locks. 
> 
>>> On Dec 3, 2019, at 6:35 PM, burak serdar  wrote:
>>> 
>>> On Tue, Dec 3, 2019 at 5:21 PM Liam Breck  wrote:
>>> 
>>> I have a problem that is trivially solved via
>>> 
>>> door sync.RWMutex
>>> 
>>> func Reader() T {
>>>  if !door.TryRLock() { // missing in stdlib :-(
>>> return busy
>>>  }
>>>  defer door.RUnlock()
>>>  ...
>>> }
>>> 
>>> func Writer() {
>>>  door.Lock()
>>>  defer door.Unlock()
>>>  ...
>>> }
>>> 
>>> How does one achieve this in Go?
>> 
>> Two locks and a bool?
>> 
>> var door=sync.Mutex{}
>> var x=sync.Mutex{}
>> var b bool
>> 
>> func trylock() bool {
>> x.Lock()
>> if b {
>> x.Unlock()
>> return false
>> }
>> b=true
>> door.Lock()
>> x.Unlock()
>> return true
>> }
>> 
>> unlock:
>> 
>> x.Lock()
>> b=false
>> door.Unlock()
>> x.Unlock()
>> 
>> 
>> 
>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAMV2RqrDeBNhkeswg%2BhdCf1kSzMEJduota%3D6UrNq4z2PQRtzEQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/04FB6D53-9937-4AA4-B640-0D276640718B%40ix.netcom.com.

Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

[Robert Engels]

Can’t code on the phone but a cas on the atomic then take the lock. But if you 
never need to block you don’t even need the lock. 

> On Dec 3, 2019, at 6:52 PM, Liam Breck  wrote:
> 
> 
> Which looks like...?
> 
>> On Tue, Dec 3, 2019, 4:50 PM Robert Engels  wrote:
>> I would use an atomic and a lock instead of two locks. 
>> 
>> > On Dec 3, 2019, at 6:35 PM, burak serdar  wrote:
>> > 
>> > On Tue, Dec 3, 2019 at 5:21 PM Liam Breck  wrote:
>> >> 
>> >> I have a problem that is trivially solved via
>> >> 
>> >> door sync.RWMutex
>> >> 
>> >> func Reader() T {
>> >>   if !door.TryRLock() { // missing in stdlib :-(
>> >>  return busy
>> >>   }
>> >>   defer door.RUnlock()
>> >>   ...
>> >> }
>> >> 
>> >> func Writer() {
>> >>   door.Lock()
>> >>   defer door.Unlock()
>> >>   ...
>> >> }
>> >> 
>> >> How does one achieve this in Go?
>> > 
>> > Two locks and a bool?
>> > 
>> > var door=sync.Mutex{}
>> > var x=sync.Mutex{}
>> > var b bool
>> > 
>> > func trylock() bool {
>> > x.Lock()
>> > if b {
>> >  x.Unlock()
>> >  return false
>> > }
>> > b=true
>> > door.Lock()
>> > x.Unlock()
>> > return true
>> > }
>> > 
>> > unlock:
>> > 
>> > x.Lock()
>> > b=false
>> > door.Unlock()
>> > x.Unlock()
>> > 
>> > 
>> > 
>> > 
>> >> 
>> >> --
>> >> You received this message because you are subscribed to the Google Groups 
>> >> "golang-nuts" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send an 
>> >> email to golang-nuts+unsubscr...@googlegroups.com.
>> >> To view this discussion on the web visit 
>> >> https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com.
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google Groups 
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to golang-nuts+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/golang-nuts/CAMV2RqrDeBNhkeswg%2BhdCf1kSzMEJduota%3D6UrNq4z2PQRtzEQ%40mail.gmail.com.
>> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/51F93012-4B18-44B3-B535-E4BC1FD9%40ix.netcom.com.

Re: [go-nuts] Workaround for missing RWMutex.Try*Lock()

[Robert Engels]

I would use an atomic and a lock instead of two locks. 

> On Dec 3, 2019, at 6:35 PM, burak serdar  wrote:
> 
> On Tue, Dec 3, 2019 at 5:21 PM Liam Breck  wrote:
>> 
>> I have a problem that is trivially solved via
>> 
>> door sync.RWMutex
>> 
>> func Reader() T {
>>   if !door.TryRLock() { // missing in stdlib :-(
>>  return busy
>>   }
>>   defer door.RUnlock()
>>   ...
>> }
>> 
>> func Writer() {
>>   door.Lock()
>>   defer door.Unlock()
>>   ...
>> }
>> 
>> How does one achieve this in Go?
> 
> Two locks and a bool?
> 
> var door=sync.Mutex{}
> var x=sync.Mutex{}
> var b bool
> 
> func trylock() bool {
> x.Lock()
> if b {
>  x.Unlock()
>  return false
> }
> b=true
> door.Lock()
> x.Unlock()
> return true
> }
> 
> unlock:
> 
> x.Lock()
> b=false
> door.Unlock()
> x.Unlock()
> 
> 
> 
> 
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAKvHMgTO%3DxfFQ_u7aO9UE-1vHHEKmdhr47sro2mnp6DkEb6mPA%40mail.gmail.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAMV2RqrDeBNhkeswg%2BhdCf1kSzMEJduota%3D6UrNq4z2PQRtzEQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/70F164B9-88EB-414F-9E61-378C247BF957%40ix.netcom.com.

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Robert Engels]

Sorry, I hadn’t looked at the code, but after reviewing doesn’t your code not 
detect the case where a value was skipped ? (... not that it could happen - but 
for completeness of the robust test)

> On Dec 1, 2019, at 10:44 PM, Michael Jones  wrote:
> 
> not necessary as the testing and updating is only done in one place by one 
> the main goroutine. 
> 
>> On Sun, Dec 1, 2019 at 7:46 PM Robert Engels  wrote:
>> The updating of the bit array if shared needs to atomic as well, probably 
>> with a read and cas.  
>> 
>>> On Dec 1, 2019, at 9:19 PM, Liam  wrote:
>>> 
>>> 
>>> Oh you've allocated a bit array for every value in the test range, then 
>>> checked for gaps in it?
>>> 
>>>> On Sunday, December 1, 2019 at 2:21:55 PM UTC-8, Michael Jones wrote:
>>>> Oh! That's just a bit per integer in the test range 0..total-1. Since Go 
>>>> (and everything else) lacks a bit type, I just type such code 
>>>> automatically. Bytes hold 8 bits. Array size must be rounded up, so
>>>> 
>>>> a := make([]byte, (total+8-1)/8)
>>>> 
>>>> array index for test integer n is n/8, so "n>>3"
>>>> 
>>>> bit index for the j-th bit, counting up from 0 for the 1's place is "1<>>> 
>>>> j is n%8, so "n&(8-1)"
>>>> 
>>>> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>>>> 
>>>> a[n>>3] & mask != 0
>>>> 
>>>> to set it is 
>>>> 
>>>> a[n>>3] |= mask
>>>> 
>>>> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using 
>>>> 64-bit ints they become 6 and 64. 
>>>> 
>>>>> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>>>>> I wrote a less-sophisticated version of your test, then realized I'd 
>>>>> misspent my time; all I needed was to change the atomic.Add*() to a 
>>>>> mutex-protected counter, and see whether my app still failed; it did.
>>>>> 
>>>>> But since you took the trouble, I read your code, and would like to 
>>>>> understand your collision detector. Could you explain this bit?
>>>>> 
>>>>> for _, v := range a {
>>>>>   mask := byte(1 << (v & (8 - 1))) 
>>>>>   index := v >> 3
>>>>> 
>>>>>   if tally[index]&mask != 0 { ... }
>>>>>   ...
>>>>> }
>>>>> 
>>>>>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>>>> As a follow-up, some more timing:
>>>>>> 
>>>>>> 47088064 atomic increments/sec (my original email above for heavy 
>>>>>> synchronization conflict incrementing)
>>>>>> 
>>>>>> 142049067 atomic increments/sec when each goroutine has its own atomic 
>>>>>> update target. (Not testing global synchronization/mutex, just the 
>>>>>> overhead of congested vs not.)
>>>>>> 
>>>>>> 426232527 ordinary "x++" increments in the workers.
>>>>>> 
>>>>>> General idea to remember:
>>>>>> 
>>>>>> Atomic increment is ~3x slower than simple add when uncontested.
>>>>>> Highly contested atomic increment is ~3x closer than uncontested, 
>>>>>> therefore ~9x-10x slower than simple add.
>>>>>> 
>>>>>> 10x is not insignificant, but is nevertheless remarkable for a reliable 
>>>>>> atomic operation. This was once, "back in the day", a remarkably 
>>>>>> expensive operation, an a feat of genius to accomplish (Dekker's 
>>>>>> Algorithm). That it is now just a number-of-fingers cycles is fantastic 
>>>>>> progress!
>>>>>> 
>>>>>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones  
>>>>>>> wrote:
>>>>>>> Liam,
>>>>>>> 
>>>>>>> I just wrote a little stress test program for you. Maybe it will make 
>>>>>>> you less stressed. ;-)
>>>>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>>>> 
>>>>>>> 4 CPU 2016 MacBook Pro:
>>>>>>> celeste:atom mtj$ go run main.go
>>>>>>> 32 concurrent workers
>>>>>>> 12

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Robert Engels]

The updating of the bit array if shared needs to atomic as well, probably with 
a read and cas.  

> On Dec 1, 2019, at 9:19 PM, Liam  wrote:
> 
> 
> Oh you've allocated a bit array for every value in the test range, then 
> checked for gaps in it?
> 
>> On Sunday, December 1, 2019 at 2:21:55 PM UTC-8, Michael Jones wrote:
>> Oh! That's just a bit per integer in the test range 0..total-1. Since Go 
>> (and everything else) lacks a bit type, I just type such code automatically. 
>> Bytes hold 8 bits. Array size must be rounded up, so
>> 
>> a := make([]byte, (total+8-1)/8)
>> 
>> array index for test integer n is n/8, so "n>>3"
>> 
>> bit index for the j-th bit, counting up from 0 for the 1's place is "1<> 
>> j is n%8, so "n&(8-1)"
>> 
>> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>> 
>> a[n>>3] & mask != 0
>> 
>> to set it is 
>> 
>> a[n>>3] |= mask
>> 
>> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using 
>> 64-bit ints they become 6 and 64. 
>> 
>>> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>>> I wrote a less-sophisticated version of your test, then realized I'd 
>>> misspent my time; all I needed was to change the atomic.Add*() to a 
>>> mutex-protected counter, and see whether my app still failed; it did.
>>> 
>>> But since you took the trouble, I read your code, and would like to 
>>> understand your collision detector. Could you explain this bit?
>>> 
>>> for _, v := range a {
>>>   mask := byte(1 << (v & (8 - 1))) 
>>>   index := v >> 3
>>> 
>>>   if tally[index]&mask != 0 { ... }
>>>   ...
>>> }
>>> 
>>>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>> As a follow-up, some more timing:
>>>> 
>>>> 47088064 atomic increments/sec (my original email above for heavy 
>>>> synchronization conflict incrementing)
>>>> 
>>>> 142049067 atomic increments/sec when each goroutine has its own atomic 
>>>> update target. (Not testing global synchronization/mutex, just the 
>>>> overhead of congested vs not.)
>>>> 
>>>> 426232527 ordinary "x++" increments in the workers.
>>>> 
>>>> General idea to remember:
>>>> 
>>>> Atomic increment is ~3x slower than simple add when uncontested.
>>>> Highly contested atomic increment is ~3x closer than uncontested, 
>>>> therefore ~9x-10x slower than simple add.
>>>> 
>>>> 10x is not insignificant, but is nevertheless remarkable for a reliable 
>>>> atomic operation. This was once, "back in the day", a remarkably expensive 
>>>> operation, an a feat of genius to accomplish (Dekker's Algorithm). That it 
>>>> is now just a number-of-fingers cycles is fantastic progress!
>>>> 
>>>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones  wrote:
>>>>> Liam,
>>>>> 
>>>>> I just wrote a little stress test program for you. Maybe it will make you 
>>>>> less stressed. ;-)
>>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>> 
>>>>> 4 CPU 2016 MacBook Pro:
>>>>> celeste:atom mtj$ go run main.go
>>>>> 32 concurrent workers
>>>>> 128 batches of 1048576 atomic increments, 134217728 total increments
>>>>> 2.850 seconds elapsed, 47088064 atomic increments/sec
>>>>> 0 collisions
>>>>> 
>>>>> 18 CPU 2019 iMacPro:
>>>>> plum:atom mtj$ go run main.go
>>>>> 32 concurrent workers
>>>>> 128 batches of 1048576 atomic increments, 134217728 total increments
>>>>> 2.730 seconds elapsed, 49167382 atomic increments/sec
>>>>> 0 collisions
>>>>> 
>>>>> Exhaustive demonstration is no proof, but changing the parameters here 
>>>>> may increase your comfort.
>>>>> 
>>>>> Michael
>>>>> 
>>>>>> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels  
>>>>>> wrote:
>>>>>> If this was broken I think a lot of things would break. 
>>>>>> 
>>>>>>>> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
>>>>>>>> 
>>>>>>> 
>>>>>>>

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[robert engels]

I’d prefer v / 8 over v >> 3 - provides more context in my opinion. The 
compiler will change to right shift if more efficient anyway.

> On Dec 1, 2019, at 4:21 PM, Michael Jones  wrote:
> 
> Oh! That's just a bit per integer in the test range 0..total-1. Since Go (and 
> everything else) lacks a bit type, I just type such code automatically. Bytes 
> hold 8 bits. Array size must be rounded up, so
> 
> a := make([]byte, (total+8-1)/8)
> 
> array index for test integer n is n/8, so "n>>3"
> 
> bit index for the j-th bit, counting up from 0 for the 1's place is "1< 
> j is n%8, so "n&(8-1)"
> 
> if mask=1<<(n&(8-1)) then one can test if the bit is set with
> 
> a[n>>3] & mask != 0
> 
> to set it is 
> 
> a[n>>3] |= mask
> 
> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using 
> 64-bit ints they become 6 and 64. 
> 
> On Sat, Nov 30, 2019 at 7:06 PM Liam  <mailto:networkimp...@gmail.com>> wrote:
> I wrote a less-sophisticated version of your test, then realized I'd misspent 
> my time; all I needed was to change the atomic.Add*() to a mutex-protected 
> counter, and see whether my app still failed; it did.
> 
> But since you took the trouble, I read your code, and would like to 
> understand your collision detector. Could you explain this bit?
> 
> for _, v := range a {
>   mask := byte(1 << (v & (8 - 1))) 
>   index := v >> 3
> 
>   if tally[index]&mask != 0 { ... }
>   ...
> }
> 
> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
> As a follow-up, some more timing:
> 
> 47088064 atomic increments/sec (my original email above for heavy 
> synchronization conflict incrementing)
> 
> 142049067 atomic increments/sec when each goroutine has its own atomic update 
> target. (Not testing global synchronization/mutex, just the overhead of 
> congested vs not.)
> 
> 426232527 ordinary "x++" increments in the workers.
> 
> General idea to remember:
> 
> Atomic increment is ~3x slower than simple add when uncontested.
> Highly contested atomic increment is ~3x closer than uncontested, therefore 
> ~9x-10x slower than simple add.
> 
> 10x is not insignificant, but is nevertheless remarkable for a reliable 
> atomic operation. This was once, "back in the day", a remarkably expensive 
> operation, an a feat of genius to accomplish (Dekker's Algorithm 
> <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That it is now just a 
> number-of-fingers cycles is fantastic progress!
> 
> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones > wrote:
> Liam,
> 
> I just wrote a little stress test program for you. Maybe it will make you 
> less stressed. ;-)
> https://play.golang.org/p/5_7Geyczd1V <https://play.golang.org/p/5_7Geyczd1V>
> 
> 4 CPU 2016 MacBook Pro:
> celeste:atom mtj$ go run main.go
> 32 concurrent workers
> 128 batches of 1048576 atomic increments, 134217728 total increments
> 2.850 seconds elapsed, 47088064 atomic increments/sec
> 0 collisions
> 
> 18 CPU 2019 iMacPro:
> plum:atom mtj$ go run main.go
> 32 concurrent workers
> 128 batches of 1048576 atomic increments, 134217728 total increments
> 2.730 seconds elapsed, 49167382 atomic increments/sec
> 0 collisions
> 
> Exhaustive demonstration is no proof, but changing the parameters here may 
> increase your comfort.
> 
> Michael
> 
> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels > wrote:
> If this was broken I think a lot of things would break. 
> 
>> On Nov 30, 2019, at 1:56 PM, Liam > wrote:
>> 
>> 
>> The stress test for my app fails frequently with what looks like a collision 
>> in atomic.AddUint64() results, so I wondered whether I had misunderstood 
>> atomic-add.
>> 
>> So far I can't reproduce it with a small program, so I've probably 
>> misunderstood my app :-)
>> 
>> On Friday, November 29, 2019 at 6:41:39 PM UTC-8, Kurtis Rader wrote:
>> On Fri, Nov 29, 2019 at 6:21 PM Liam > wrote:
>> Does atomic.AddInt32(&x, 1) always yield unique values for concurrent 
>> callers?
>> 
>> I'm guessing not, because (I think) I'm seeing that two callers get x+2, 
>> neither gets x+1.
>> 
>> That shouldn't happen, AFAICT. Can you share the code where the incorrect 
>> behavior is occurring? Or, preferably, a simple reproducer program?
>>   
>> Is there a way to generate unique values with pkg atomic, or is a mutex 
>> required?
>> 
>> Keep in mind that atomic.AddInt32() has the usual two's-complement 

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Robert Engels]

If this was broken I think a lot of things would break. 

> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
> 
> 
> The stress test for my app fails frequently with what looks like a collision 
> in atomic.AddUint64() results, so I wondered whether I had misunderstood 
> atomic-add.
> 
> So far I can't reproduce it with a small program, so I've probably 
> misunderstood my app :-)
> 
>> On Friday, November 29, 2019 at 6:41:39 PM UTC-8, Kurtis Rader wrote:
>>> On Fri, Nov 29, 2019 at 6:21 PM Liam  wrote:
>> 
>>> Does atomic.AddInt32(&x, 1) always yield unique values for concurrent 
>>> callers?
>>> 
>>> I'm guessing not, because (I think) I'm seeing that two callers get x+2, 
>>> neither gets x+1.
>> 
>> That shouldn't happen, AFAICT. Can you share the code where the incorrect 
>> behavior is occurring? Or, preferably, a simple reproducer program?
>>   
>>> Is there a way to generate unique values with pkg atomic, or is a mutex 
>>> required?
>> 
>> Keep in mind that atomic.AddInt32() has the usual two's-complement  overflow 
>> semantics. If all you want is a generation counter you really should be 
>> using a uint32 and atomic.AddUint32(). Also, depending on your preferences 
>> and performance considerations you might find it preferable to use a channel 
>> that holds a single int, or small number of ints, that is fed by a producer 
>> goroutine and consumed by any context needing a uniq ID. That makes it 
>> easier to abstract the generation of "unique" ints so that they satisfy 
>> other constraints (e.g., they must be even, odd, prime, etc.).
>> 
>> -- 
>> Kurtis Rader
>> Caretaker of the exceptional canines Junior and Hank
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/4f62dfff-6895-4aaa-9f0d-b635d5ba7ea7%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C7B99DEA-D183-44EF-9EDA-0B1841AB9DE5%40ix.netcom.com.

Re: [go-nuts] C variadic macro equivalent in Golang?

[Robert Engels]

Make the type interface{} and you can pass anything and use reflect

> On Nov 30, 2019, at 2:08 PM, minfo...@arcor.de wrote:
> 
> 
> C allows comma-separated argument lists via variadic macros. Whereas AFAIK 
> golang allows only variadic arguments of the same type.
> (Please excuse if I am wrong, I am considering myself a golang newbie)
> 
> C demo program that prints:   -- -- 1 and 1 1 and 1
> 
> // ## C variadic macro test
> 
> #include 
> 
> #define PR(...) if(df){printf(__VA_ARGS__);\
> if(ef)fprintf(stderr,__VA_ARGS__);}
> 
> // Flags
> int df=1, ef=1;
> 
> int main(void) {
>   PR("-- ");
>   PR("%d and %d ",df,ef);
> }
> 
> // ## End
> 
> How would one implement an equivalent PR(...) "function" in Go ???
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/e36eb032-ffbc-4b26-8c41-f76aa6dcdd00%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/E0C24B80-31F4-4A6C-AEAD-DB0FA533C595%40ix.netcom.com.

Re: [go-nuts] runtime: panic in net/http.(*Server).Serve

[Robert Engels]

Just because you print all of the parameters and they’re ok doesn’t mean there 
isn’t any corruption. Sorry I can’t be of more help. 

> On Nov 26, 2019, at 3:22 AM, 黃翔蔚  wrote:
> 
> 
> Hi Robert,
> Thanks for the explanation. 
> There is only one function call from GO to CPP in the program.
> I printf all parameters in the CPP function.
> There is no memory corruption inside.
> If you have any other troubleshooting suggestions, I would appreciate it.
> 
> Walter
> 
> Robert Engels於 2019年11月26日星期二 UTC+8下午2時31分55秒寫道：
>> 
>> It looks like the race detector outputs when it detects a race, not at 
>> program end, so it doesn’t apply in your case. 
>> 
>> So, I would look into cgo and other unsafe usages. 
>> 
>>> On Nov 26, 2019, at 12:18 AM, 黃翔蔚  wrote:
>>> 
>>> Hi Robert,
>>> I don't understand what you mean. Could you please explain more detail?
>>> Thanks.
>>> 
>>> Walter
>>> 
>>> Robert Engels於 2019年11月26日星期二 UTC+8上午9時16分42秒寫道：
>>>> 
>>>> I think with -race you want to exit before it crashes. After running a 
>>>> while. Can’t remember though. Check the docs. 
>>>> 
>>>>>> On Nov 25, 2019, at 7:10 PM, 黃翔蔚  wrote:
>>>>>> 
>>>>> 
>>>>> Hi lan,
>>>>> Thanks for your reply.
>>>>> I built the program with -race option. There is no news in crash dump.
>>>>> I will review cgo portion again. Thanks again.
>>>>> 
>>>>> Walter
>>>>> 
>>>>> Ian Lance Taylor於 2019年11月26日星期二 UTC+8上午3時12分06秒寫道：
>>>>>> 
>>>>>> On Mon, Nov 25, 2019 at 9:23 AM 黃翔蔚  wrote: 
>>>>>> > 
>>>>>> > My program will get panic issue from time to time. 
>>>>>> > I don't have procedure to reproduce this issue.
>>>>>> > So far, the issue is happen on 10/23, 10/25, 10/30, 11/14 and 11/22. 
>>>>>> > In the last try, I capture packets by wireshark, it seems no packet 
>>>>>> > incoming when panic. 
>>>>>> > I also try go build -race. There is no useful clues in crash dump. 
>>>>>> > Any suggestion is welcome. 
>>>>>> 
>>>>>> The most common cause for this kind of problem is memory corruption. 
>>>>>> Run your program under the race detector.  Look closely at any uses of 
>>>>>> unsafe or cgo. 
>>>>>> 
>>>>>> Ian 
>>>>> 
>>>>> -- 
>>>>> You received this message because you are subscribed to the Google Groups 
>>>>> "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>>>> email to golan...@googlegroups.com.
>>>>> To view this discussion on the web visit 
>>>>> https://groups.google.com/d/msgid/golang-nuts/c68e43ec-11a5-4d0c-9eac-b31d8fb91d7a%40googlegroups.com.
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/a25c9e13-e92c-4802-9ea9-15d3c7aeeba0%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/2ea34766-e924-471a-b228-442cfa8fe43b%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/D64420D2-CF6C-46A6-AB90-A3138A78D7DF%40ix.netcom.com.

Re: [go-nuts] runtime: panic in net/http.(*Server).Serve

[Robert Engels]

It looks like the race detector outputs when it detects a race, not at program 
end, so it doesn’t apply in your case. 

So, I would look into cgo and other unsafe usages. 

> On Nov 26, 2019, at 12:18 AM, 黃翔蔚  wrote:
> 
> Hi Robert,
> I don't understand what you mean. Could you please explain more detail?
> Thanks.
> 
> Walter
> 
> Robert Engels於 2019年11月26日星期二 UTC+8上午9時16分42秒寫道：
>> 
>> I think with -race you want to exit before it crashes. After running a 
>> while. Can’t remember though. Check the docs. 
>> 
>>> On Nov 25, 2019, at 7:10 PM, 黃翔蔚  wrote:
>>> 
>>> 
>>> Hi lan,
>>> Thanks for your reply.
>>> I built the program with -race option. There is no news in crash dump.
>>> I will review cgo portion again. Thanks again.
>>> 
>>> Walter
>>> 
>>> Ian Lance Taylor於 2019年11月26日星期二 UTC+8上午3時12分06秒寫道：
>>>> 
>>>> On Mon, Nov 25, 2019 at 9:23 AM 黃翔蔚  wrote: 
>>>> > 
>>>> > My program will get panic issue from time to time. 
>>>> > I don't have procedure to reproduce this issue. 
>>>> > So far, the issue is happen on 10/23, 10/25, 10/30, 11/14 and 11/22. 
>>>> > In the last try, I capture packets by wireshark, it seems no packet 
>>>> > incoming when panic. 
>>>> > I also try go build -race. There is no useful clues in crash dump. 
>>>> > Any suggestion is welcome. 
>>>> 
>>>> The most common cause for this kind of problem is memory corruption. 
>>>> Run your program under the race detector.  Look closely at any uses of 
>>>> unsafe or cgo. 
>>>> 
>>>> Ian 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/c68e43ec-11a5-4d0c-9eac-b31d8fb91d7a%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/a25c9e13-e92c-4802-9ea9-15d3c7aeeba0%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7458BEE7-3CFB-4122-B88D-CD5FBB4C0A9B%40ix.netcom.com.

Re: [go-nuts] runtime: panic in net/http.(*Server).Serve

[Robert Engels]

I think with -race you want to exit before it crashes. After running a while. 
Can’t remember though. Check the docs. 

> On Nov 25, 2019, at 7:10 PM, 黃翔蔚  wrote:
> 
> 
> Hi lan,
> Thanks for your reply.
> I built the program with -race option. There is no news in crash dump.
> I will review cgo portion again. Thanks again.
> 
> Walter
> 
> Ian Lance Taylor於 2019年11月26日星期二 UTC+8上午3時12分06秒寫道：
>> 
>> On Mon, Nov 25, 2019 at 9:23 AM 黃翔蔚  wrote: 
>> > 
>> > My program will get panic issue from time to time. 
>> > I don't have procedure to reproduce this issue. 
>> > So far, the issue is happen on 10/23, 10/25, 10/30, 11/14 and 11/22. 
>> > In the last try, I capture packets by wireshark, it seems no packet 
>> > incoming when panic. 
>> > I also try go build -race. There is no useful clues in crash dump. 
>> > Any suggestion is welcome. 
>> 
>> The most common cause for this kind of problem is memory corruption. 
>> Run your program under the race detector.  Look closely at any uses of 
>> unsafe or cgo. 
>> 
>> Ian 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/c68e43ec-11a5-4d0c-9eac-b31d8fb91d7a%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/09DDD2E0-EC66-4EFA-B5F9-F5D83C15D720%40ix.netcom.com.

Re: [go-nuts] [ANN] Golang Oriented to Aspects (goa) v0.0.1 is released!!!

[Robert Engels]

I think a “how it works” and “usages” in the documentation would be helpful. 
Great work. 

> On Nov 25, 2019, at 5:19 PM, Dimas Prawira  
> wrote:
> 
> 
> Nice... #clap
> 
> BR
> 
> 
>> On Tue, Nov 26, 2019, 4:47 AM Iván Corrales Solera 
>>  wrote:
>> Hey folks!
>> 
>> I am so happy yo announce the first release of Goa. Goa is a library that 
>> will drive you to the AOP (Aspect Oriented Paradigm) world!
>> 
>> Even though I still have to work hard on this library, I would appreciate 
>> some feedback for you guys!.
>> 
>> The full documentation can be found here 
>> Source ode https://github.com/wesovilabs/goa
>> Full documentation http://wesovilabs.github.io/goa 
>> 
>> Thank you so much for your time! and I hope you enjoy it!
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/7b0347e8-e7c1-4dab-afcb-9608573d7d2b%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CA%2Bp%2BMUeMP54Lv636hNE5i4_M4u1JCEsJaW-NFnf%3DgjMGiwGvTw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/39F094A1-20ED-4611-A157-47103FD48090%40ix.netcom.com.

Re: [go-nuts] looking for a snippet to get started with Tcell or Termbox-go

[Robert Engels]

Use gcui. Check out github.com/robaho/go-trader for an example. 

> On Nov 25, 2019, at 11:23 AM, serge.hulne.bl...@gmail.com wrote:
> 
> 
> Hi,
> 
> I am looking for the easiest way to emulate ncurses using Go.
> 
> More precisely, I am looking for a snippet of code (an example basically) on 
> how to do the following:
> 
> "put a given character directly on a location (x,y) on a terminal" using 
> either Tcell or Termbox-go
> 
> 
> 
> or in pseudo-code:
> 
> put("X", x, y)
> 
> 
> 
> My aim is to build a "Game-of-life" like application. To this end I'll need 
> to e able to address randomly locations on a terminal.
> 
> PS: I am trying to avoid goncurses, for portability reasons.
> 
> Thanks in advance,
> Any help will be greatly appreciated.
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/a2b17fb3-0774-44de-86bb-2937b06f2ca0%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/D4E80143-D8DD-4EBE-94EC-0BC9AF5D4E77%40ix.netcom.com.

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

I don't think anyone was mocking the question or thought it was ridiculous.-Original Message-
From: Robert Johnstone 
Sent: Nov 22, 2019 8:40 AM
To: golang-nuts 
Subject: Re: [go-nuts] Enforce immutability through static analysis

This comment is a little unfair.  There was at one time efforts to allow const as part of the type system.  I believe that the specific motivation was to allow []const byte to ease conversions are eliminate conversions from strings.  The current duplication of bytes and strings packages is a bit of a smell.  If you had instead written an example for an interface, the request would not seem so ridiculous:  interface X {    const f() int  }(Not advocating a change to the language, just pointing out that the question should not be mocked)On Thursday, 21 November 2019 13:01:09 UTC-5, burak serdar  wrote:The way I read the original post what is being asked is, is it
possible to have the Go-equivalent of the following C++ code:

class X{
  public:
  virtual int f() const =0;
}


>
>
>
> -Original Message-
> >From: burak serdar <bse...@computer.org>
> >Sent: Nov 21, 2019 11:34 AM
> >To: Robert Engels <ren...@ix.netcom.com>
> >Cc: advand...@gmail.com, golang-nuts <golan...@googlegroups.com>
> >Subject: Re: [go-nuts] Enforce immutability through static analysis
> >
> >On Thu, Nov 21, 2019 at 10:25 AM Robert Engels <ren...@ix.netcom.com> wrote:
> >>
> >> I don't think we are talking about the same thing. You can certainly code an immutable object - just don't export any methods that mutate the object, nor export ANY fields.
> >
> >Correct, we're talking about different things. The question is not
> >whether you can write an immutable object (yes you can), it is whether
> >there is a way to enforce the immutability of the receiver of a
> >method.
> >
> >If the method is exported and if the receiver contains pointers, there
> >can be no guarantee that the method will not modify values reachable
> >from the copy of the receiver.
> >
> >>
> >>
> >>
> >>
> >> -Original Message-
> >> >From: burak serdar <bse...@computer.org>
> >> >Sent: Nov 21, 2019 11:09 AM
> >> >To: Robert Engels <ren...@ix.netcom.com>
> >> >Cc: advand...@gmail.com, golang-nuts <golan...@googlegroups.com>
> >> >Subject: Re: [go-nuts] Enforce immutability through static analysis
> >> >
> >> >On Thu, Nov 21, 2019 at 10:05 AM Robert Engels <ren...@ix.netcom.com> wrote:
> >> >>
> >> >> To clarify - the author of the package enforces immutability. With Go’s design this can be a simple comment on the field. The package shouldn’t be that large where this doesn’t work.
> >> >
> >> >The original problem remains: there is no way to enforce an immutable receiver.
> >> >
> >> >>
> >> >> > On Nov 21, 2019, at 10:58 AM, Robert Engels <ren...@ix.netcom.com> wrote:
> >> >> >
> >> >> > 
> >> >> > Correct, but if the receiver method is mutating it, then it is not an immutable object.
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > -Original Message-
> >> >> >> From: burak serdar <bse...@computer.org>
> >> >> >> Sent: Nov 21, 2019 10:53 AM
> >> >> >> To: Robert Engels <ren...@ix.netcom.com>
> >> >> >> Cc: advand...@gmail.com, golang-nuts <golan...@googlegroups.com>
> >> >> >> Subject: Re: [go-nuts] Enforce immutability through static analysis
> >> >> >>
> >> >> >>> On Thu, Nov 21, 2019 at 9:49 AM Robert Engels <ren...@ix.netcom.com> wrote:
> >> >> >>>
> >> >> >>> They can't unless the instance field is exported. Just hide it via encapsulation with accessors.
> >> >> >>
> >> >> >> Can't do that with a receiver. All methods of a type are in the same
> >> >> >> package as the type, so all fields are visible to the receiver.
> >> >> >>
> >> >> >>
> >> >> >>>
> >> >> >>> -Original Message-
> >> >> >>> From: advand...@gmail.com
> >> >> >>> Sent: Nov 21, 2019 10:15 AM
> >> >> >>> To: golang-nuts
> >> >> >>> Subject: [go-nuts] Enforce immutability through static

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

I would just copy and paste the stdlib pipe.go and change the ctor to take a 
size for the channel. I’m pretty sure that is all that is needed to fix the 
problem. 

> On Nov 21, 2019, at 11:18 PM, Marcin Romaszewicz  wrote:
> 
> 
> I am in fact replacing an io.Pipe implementation, because I need to buffer 
> some data. io.Pipe doesn't buffer, it just matches up read and writes with 
> each other.
> 
> What I'm effectively doing is producing chunks of data at a certain rate from 
> one server, and forwarding them to another. Due to the latency of issuing the 
> read from the server, I need to do some pre-fetch into a buffer, but I don't 
> want to prefetch to much.
> 
> With the pipe implementation, my reader had many stalls waiting for the 
> server to produce the next  chunk. The server can produce data a lot quicker 
> than the reader can read, however, setup time is high. It's a complicated 
> mess :)
> 
>> On Thu, Nov 21, 2019 at 8:37 PM Robert Engels  wrote:
>> The OP specifically requested io.Reader/Writer interfaces. 
>> 
>> A pipe is what he wants. Not a ring buffer. (A pipe essentially has a ring 
>> buffer in the implementation though). 
>> 
>> > On Nov 21, 2019, at 10:20 PM, Dan Kortschak  wrote:
>> > 
>> > There is this: https://godoc.org/bitbucket.org/ausocean/utils/ring
>> > 
>> > It has been used in production fairly extensively.
>> > 
>> >> On Thu, 2019-11-21 at 19:47 -0800, Marcin Romaszewicz wrote:
>> >> Hi All,
>> >> 
>> >> Before I reinvent the wheel, and because this wheel is particularly
>> >> tricky
>> >> to get right, I was wondering if anyone was aware of a a library
>> >> providing
>> >> something like this
>> >> 
>> >> - conforms to io.Reader
>> >> - conforms to io.Writer
>> >> - Contains a buffer of fixed size, say, 64MB. If you try to write
>> >> when the
>> >> buffer is too full, write blocks. When you try to read from an empty
>> >> one,
>> >> read blocks.
>> >> 
>> >> This describes the behavior of make(chan byte, 64 * MB), however, it
>> >> doesn't seem to be efficient to do this with a channel. Say I'm
>> >> transferring a few hundred GB via this mechanism. A chan of byte
>> >> would need
>> >> a few hundred billion byte writes, and a few hundred billion reads.
>> >> Doesn't
>> >> sound like it could be efficient at all. You can't implement this
>> >> with a
>> >> channel of []byte, because you'd violate your buffering limit if you
>> >> pass
>> >> too large of a byte array, so you'd have to chop things up into
>> >> blocks, but
>> >> maybe that's simpler than a full fledged blocking ring buffer.
>> >> 
>> >> Anyhow, I've implemented such things in various OS level plumbing, so
>> >> I
>> >> know I can do it in Go much more easily, just hoping to avoid it :)
>> >> 
>> >> Thanks for any advice,
>> >> -- Marcin
>> >> 
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google Groups 
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to golang-nuts+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/golang-nuts/152a954eaa3eea02514e71bb142904480241ad6c.camel%40kortschak.io.
>> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C1587EA0-9752-4AE5-A232-BB5DCCC2343E%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

Yes, I already addressed this but I know people often reply as they read. 

This is not really a pipe in the traditional posix sense. A bad name, or bad 
implementation imo. 

> On Nov 21, 2019, at 10:53 PM, Jamil Djadala  wrote:
> 
> On Thu, 21 Nov 2019 22:37:37 -0600
> Robert Engels  wrote:
> 
>> The OP specifically requested io.Reader/Writer interfaces. 
>> 
>> A pipe is what he wants. Not a ring buffer. (A pipe essentially has a
>> ring buffer in the implementation though). 
> 
> from https://golang.org/pkg/io/#Pipe :
> 
> The data is copied directly from the Write to the corresponding Read
> (or Reads); there is no internal buffering
> 
> and looking in implementation, it seems that internally Pipe is using
> channel.
> 
> -- 
> Jamil Djadala
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/20191122065315.78316d02%40wolf.home.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/AC4A7316-7473-406B-8B4A-17609E836E20%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

Not sure why the go pipe doesn’t take a size for the inner channel to make the 
synchronous nature optional. 

> On Nov 21, 2019, at 10:44 PM, Robert Engels  wrote:
> 
> Sorry in reading the implementation of a pipe in Go, it may not have the 
> desired semantics, especially with a large write buffer since writes and 
> reads are 1 to 1 (very weird impl - this is not a traditional pipe). Dan’s 
> library is a better choice. 
> 
>> On Nov 21, 2019, at 10:37 PM, Robert Engels  wrote:
>> 
>> The OP specifically requested io.Reader/Writer interfaces. 
>> 
>> A pipe is what he wants. Not a ring buffer. (A pipe essentially has a ring 
>> buffer in the implementation though). 
>> 
>>>> On Nov 21, 2019, at 10:20 PM, Dan Kortschak  wrote:
>>> 
>>> There is this: https://godoc.org/bitbucket.org/ausocean/utils/ring
>>> 
>>> It has been used in production fairly extensively.
>>> 
>>>>> On Thu, 2019-11-21 at 19:47 -0800, Marcin Romaszewicz wrote:
>>>> Hi All,
>>>> 
>>>> Before I reinvent the wheel, and because this wheel is particularly
>>>> tricky
>>>> to get right, I was wondering if anyone was aware of a a library
>>>> providing
>>>> something like this
>>>> 
>>>> - conforms to io.Reader
>>>> - conforms to io.Writer
>>>> - Contains a buffer of fixed size, say, 64MB. If you try to write
>>>> when the
>>>> buffer is too full, write blocks. When you try to read from an empty
>>>> one,
>>>> read blocks.
>>>> 
>>>> This describes the behavior of make(chan byte, 64 * MB), however, it
>>>> doesn't seem to be efficient to do this with a channel. Say I'm
>>>> transferring a few hundred GB via this mechanism. A chan of byte
>>>> would need
>>>> a few hundred billion byte writes, and a few hundred billion reads.
>>>> Doesn't
>>>> sound like it could be efficient at all. You can't implement this
>>>> with a
>>>> channel of []byte, because you'd violate your buffering limit if you
>>>> pass
>>>> too large of a byte array, so you'd have to chop things up into
>>>> blocks, but
>>>> maybe that's simpler than a full fledged blocking ring buffer.
>>>> 
>>>> Anyhow, I've implemented such things in various OS level plumbing, so
>>>> I
>>>> know I can do it in Go much more easily, just hoping to avoid it :)
>>>> 
>>>> Thanks for any advice,
>>>> -- Marcin
>>>> 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/152a954eaa3eea02514e71bb142904480241ad6c.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/16241014-EBD3-4232-8460-FEE735918ACB%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

Sorry in reading the implementation of a pipe in Go, it may not have the 
desired semantics, especially with a large write buffer since writes and reads 
are 1 to 1 (very weird impl - this is not a traditional pipe). Dan’s library is 
a better choice. 

> On Nov 21, 2019, at 10:37 PM, Robert Engels  wrote:
> 
> The OP specifically requested io.Reader/Writer interfaces. 
> 
> A pipe is what he wants. Not a ring buffer. (A pipe essentially has a ring 
> buffer in the implementation though). 
> 
>> On Nov 21, 2019, at 10:20 PM, Dan Kortschak  wrote:
>> 
>> There is this: https://godoc.org/bitbucket.org/ausocean/utils/ring
>> 
>> It has been used in production fairly extensively.
>> 
>>>> On Thu, 2019-11-21 at 19:47 -0800, Marcin Romaszewicz wrote:
>>> Hi All,
>>> 
>>> Before I reinvent the wheel, and because this wheel is particularly
>>> tricky
>>> to get right, I was wondering if anyone was aware of a a library
>>> providing
>>> something like this
>>> 
>>> - conforms to io.Reader
>>> - conforms to io.Writer
>>> - Contains a buffer of fixed size, say, 64MB. If you try to write
>>> when the
>>> buffer is too full, write blocks. When you try to read from an empty
>>> one,
>>> read blocks.
>>> 
>>> This describes the behavior of make(chan byte, 64 * MB), however, it
>>> doesn't seem to be efficient to do this with a channel. Say I'm
>>> transferring a few hundred GB via this mechanism. A chan of byte
>>> would need
>>> a few hundred billion byte writes, and a few hundred billion reads.
>>> Doesn't
>>> sound like it could be efficient at all. You can't implement this
>>> with a
>>> channel of []byte, because you'd violate your buffering limit if you
>>> pass
>>> too large of a byte array, so you'd have to chop things up into
>>> blocks, but
>>> maybe that's simpler than a full fledged blocking ring buffer.
>>> 
>>> Anyhow, I've implemented such things in various OS level plumbing, so
>>> I
>>> know I can do it in Go much more easily, just hoping to avoid it :)
>>> 
>>> Thanks for any advice,
>>> -- Marcin
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/152a954eaa3eea02514e71bb142904480241ad6c.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/A67780AB-B62A-4C31-9D44-0531DF75BD38%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

The package Dan referenced will work too - but it doesn’t look different than 
using a pipe with a large write buffer. 

> On Nov 21, 2019, at 10:20 PM, Dan Kortschak  wrote:
> 
> There is this: https://godoc.org/bitbucket.org/ausocean/utils/ring
> 
> It has been used in production fairly extensively.
> 
>> On Thu, 2019-11-21 at 19:47 -0800, Marcin Romaszewicz wrote:
>> Hi All,
>> 
>> Before I reinvent the wheel, and because this wheel is particularly
>> tricky
>> to get right, I was wondering if anyone was aware of a a library
>> providing
>> something like this
>> 
>> - conforms to io.Reader
>> - conforms to io.Writer
>> - Contains a buffer of fixed size, say, 64MB. If you try to write
>> when the
>> buffer is too full, write blocks. When you try to read from an empty
>> one,
>> read blocks.
>> 
>> This describes the behavior of make(chan byte, 64 * MB), however, it
>> doesn't seem to be efficient to do this with a channel. Say I'm
>> transferring a few hundred GB via this mechanism. A chan of byte
>> would need
>> a few hundred billion byte writes, and a few hundred billion reads.
>> Doesn't
>> sound like it could be efficient at all. You can't implement this
>> with a
>> channel of []byte, because you'd violate your buffering limit if you
>> pass
>> too large of a byte array, so you'd have to chop things up into
>> blocks, but
>> maybe that's simpler than a full fledged blocking ring buffer.
>> 
>> Anyhow, I've implemented such things in various OS level plumbing, so
>> I
>> know I can do it in Go much more easily, just hoping to avoid it :)
>> 
>> Thanks for any advice,
>> -- Marcin
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/152a954eaa3eea02514e71bb142904480241ad6c.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9968BE9B-4D20-4E0C-9997-EEDB0E6E4E8B%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

The OP specifically requested io.Reader/Writer interfaces. 

A pipe is what he wants. Not a ring buffer. (A pipe essentially has a ring 
buffer in the implementation though). 

> On Nov 21, 2019, at 10:20 PM, Dan Kortschak  wrote:
> 
> There is this: https://godoc.org/bitbucket.org/ausocean/utils/ring
> 
> It has been used in production fairly extensively.
> 
>> On Thu, 2019-11-21 at 19:47 -0800, Marcin Romaszewicz wrote:
>> Hi All,
>> 
>> Before I reinvent the wheel, and because this wheel is particularly
>> tricky
>> to get right, I was wondering if anyone was aware of a a library
>> providing
>> something like this
>> 
>> - conforms to io.Reader
>> - conforms to io.Writer
>> - Contains a buffer of fixed size, say, 64MB. If you try to write
>> when the
>> buffer is too full, write blocks. When you try to read from an empty
>> one,
>> read blocks.
>> 
>> This describes the behavior of make(chan byte, 64 * MB), however, it
>> doesn't seem to be efficient to do this with a channel. Say I'm
>> transferring a few hundred GB via this mechanism. A chan of byte
>> would need
>> a few hundred billion byte writes, and a few hundred billion reads.
>> Doesn't
>> sound like it could be efficient at all. You can't implement this
>> with a
>> channel of []byte, because you'd violate your buffering limit if you
>> pass
>> too large of a byte array, so you'd have to chop things up into
>> blocks, but
>> maybe that's simpler than a full fledged blocking ring buffer.
>> 
>> Anyhow, I've implemented such things in various OS level plumbing, so
>> I
>> know I can do it in Go much more easily, just hoping to avoid it :)
>> 
>> Thanks for any advice,
>> -- Marcin
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/152a954eaa3eea02514e71bb142904480241ad6c.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/40518F48-381C-42DA-8C78-A069388E1C9F%40ix.netcom.com.

Re: [go-nuts] Is anyone aware of a blocking ring buffer implementation?

[Robert Engels]

Just use a pipe. Part of the std lib. 

> On Nov 21, 2019, at 9:47 PM, Marcin Romaszewicz  wrote:
> 
> 
> Hi All,
> 
> Before I reinvent the wheel, and because this wheel is particularly tricky to 
> get right, I was wondering if anyone was aware of a a library providing 
> something like this
> 
> - conforms to io.Reader
> - conforms to io.Writer
> - Contains a buffer of fixed size, say, 64MB. If you try to write when the 
> buffer is too full, write blocks. When you try to read from an empty one, 
> read blocks.
> 
> This describes the behavior of make(chan byte, 64 * MB), however, it doesn't 
> seem to be efficient to do this with a channel. Say I'm transferring a few 
> hundred GB via this mechanism. A chan of byte would need a few hundred 
> billion byte writes, and a few hundred billion reads. Doesn't sound like it 
> could be efficient at all. You can't implement this with a channel of []byte, 
> because you'd violate your buffering limit if you pass too large of a byte 
> array, so you'd have to chop things up into blocks, but maybe that's simpler 
> than a full fledged blocking ring buffer.
> 
> Anyhow, I've implemented such things in various OS level plumbing, so I know 
> I can do it in Go much more easily, just hoping to avoid it :)
> 
> Thanks for any advice,
> -- Marcin
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CA%2Bv29LsvbVU1oxBYz6wKO50-xNVPQ6-wRa7Dvhq4uY%3Du-FKigA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/B9251FAA-9774-4306-A356-A88F54441980%40ix.netcom.com.

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Robert Engels]

He stated "each request takes 2 secs to process" - what's involved in that is the important aspect imo.-Original Message-
From: Michael Jones 
Sent: Nov 21, 2019 2:06 PM
To: Robert Engels 
Cc: Sankar , golang-nuts 
Subject: Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

In my (past) benchmarking, I got ~3M channel send/receive operations per second on my MacBook Pro. It is faster on faster computers. 2k requests/src is much less than 3M, clearly, and the 1/1000 duty cycle suggests that you'll have 99.9% non-overhead to do your processing. This is back of the envelope thinking, but what I go through for every path explored. What should it be? How is in in fact? What explains the difference? ... that kind of thing.On Thu, Nov 21, 2019 at 11:25 AM Robert Engels <reng...@ix.netcom.com> wrote:You need to determine how well they parallelize and what the resource consumption of a request is. For example, if every request can run concurrently at 100% (not possible btw because of switching overhead), and each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs. IO can run concurrently on most modern setups, so you can essentially factor this out, less so if most of the operations are writes.Your local CPU requirements may be less if the request is then handled by a cluster (over the network, database etc), but you will still need 1000 cpus in the cluster (probably a lot more due to the network overhead).You can look at github.com/robaho/go-trader for an example of very high CPU based processing using Go and channels (and other concurrency structs).-Original Message-
From: Sankar 
Sent: Nov 21, 2019 12:30 PM
To: golang-nuts 
Subject: [go-nuts] golang multiple go routines reading from a channel and performance implications

We have a setup where we have a producer goroutine pumping in a few thousand objects into a channel (Approximately 2k requests per second). There are a configurable number of goroutines that work as consumers, consuming from this single channel. If none of the consumer threads could receive the message, the message gets discarded. Each consumer go routine takes about 2 seconds for each work to be completed, by which they will come back to read the next item in the channel. The channel is sized such that it can hold up to 10,000 messages.The code is roughly something like:producer.go:func produce() { ch <- item}func consumer() { for i:=0; i < NumberOfWorkers; i ++ {   go func() {      for _, item := range ch {         // process item      }   } () }}With this above setup, we are seeing about 40% of our messages getting dropped.So my questions are:1) In such a high velocity incoming data, will this above design work ? (Producer, Consumer Worker Threads)2) We did not go for an external middleware for saving the message and processing data later, as we are concerned about latency for now.3) Are channels bad for such an approach ? Are there any other alternate performant mechanism to achieve this in the go way ?4) Are there any sample FOSS projects that we can refer to see such performant code ? Any other book, tutorial, video or some such for these high performance Golang application development guidelines ?I am planning to do some profiling of our system to see where the performance is getting dropped, but before that I wanted to ask here, in case there are any best-known-methods that I am missing out on. Thanks.



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f8b5d9fb-d9b7-44b8-bc50-70dcb5f10cd0%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/225583457.2024.1574364299085%40wamui-scooby.atl.sa.earthlink.net.
-- Michael T. Jonesmichael.jo...@gmail.com




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1791680454.2277.1574367136562%40wamui-scooby.atl.sa.earthlink.net.

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Robert Engels]

You need to determine how well they parallelize and what the resource consumption of a request is. For example, if every request can run concurrently at 100% (not possible btw because of switching overhead), and each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs. IO can run concurrently on most modern setups, so you can essentially factor this out, less so if most of the operations are writes.Your local CPU requirements may be less if the request is then handled by a cluster (over the network, database etc), but you will still need 1000 cpus in the cluster (probably a lot more due to the network overhead).You can look at github.com/robaho/go-trader for an example of very high CPU based processing using Go and channels (and other concurrency structs).-Original Message-
From: Sankar 
Sent: Nov 21, 2019 12:30 PM
To: golang-nuts 
Subject: [go-nuts] golang multiple go routines reading from a channel and performance implications

We have a setup where we have a producer goroutine pumping in a few thousand objects into a channel (Approximately 2k requests per second). There are a configurable number of goroutines that work as consumers, consuming from this single channel. If none of the consumer threads could receive the message, the message gets discarded. Each consumer go routine takes about 2 seconds for each work to be completed, by which they will come back to read the next item in the channel. The channel is sized such that it can hold up to 10,000 messages.The code is roughly something like:producer.go:func produce() { ch <- item}func consumer() { for i:=0; i < NumberOfWorkers; i ++ {   go func() {      for _, item := range ch {         // process item      }   } () }}With this above setup, we are seeing about 40% of our messages getting dropped.So my questions are:1) In such a high velocity incoming data, will this above design work ? (Producer, Consumer Worker Threads)2) We did not go for an external middleware for saving the message and processing data later, as we are concerned about latency for now.3) Are channels bad for such an approach ? Are there any other alternate performant mechanism to achieve this in the go way ?4) Are there any sample FOSS projects that we can refer to see such performant code ? Any other book, tutorial, video or some such for these high performance Golang application development guidelines ?I am planning to do some profiling of our system to see where the performance is getting dropped, but before that I wanted to ask here, in case there are any best-known-methods that I am missing out on. Thanks.



-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f8b5d9fb-d9b7-44b8-bc50-70dcb5f10cd0%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/225583457.2024.1574364299085%40wamui-scooby.atl.sa.earthlink.net.

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

Yes, I agree. But in some sense, a developer needed to add the 'const', i.e. 
decide that the method was immutable as part of the design (and since Go 
doesn't have inheritance it is less important). Go doesn't support this through 
syntax/compiler it supports it through design philosophy - if the packages are 
well done, and right-sized, the "immutable design" is easily "enforced & 
validated". I also think this can lead to easier to understood designs - rather 
than just slapping a bunch of const declarations.

Java is in a similar boat (a bit more difficult since it has inheritance), and 
it is not a problem with proper design. There are TONS of "immutable" Java 
libraries.

Just my opinion of course :)




-Original Message-
>From: burak serdar 
>Sent: Nov 21, 2019 12:00 PM
>To: Robert Engels 
>Cc: advanderv...@gmail.com, golang-nuts 
>Subject: Re: [go-nuts] Enforce immutability through static analysis
>
>On Thu, Nov 21, 2019 at 10:39 AM Robert Engels  wrote:
>>
>>
>>
>> I'm sorry, but isn't the way you enforce immutability is that you don't code 
>> mutating exported methods? The author controls the package code. 
>> Immutability is usually only a concern outside the package, and that is 
>> clearly supported.
>
>The way I read the original post what is being asked is, is it
>possible to have the Go-equivalent of the following C++ code:
>
>class X{
>  public:
>  virtual int f() const =0;
>}
>
>
>>
>>
>>
>> -Original Message-
>> >From: burak serdar 
>> >Sent: Nov 21, 2019 11:34 AM
>> >To: Robert Engels 
>> >Cc: advanderv...@gmail.com, golang-nuts 
>> >Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >
>> >On Thu, Nov 21, 2019 at 10:25 AM Robert Engels  
>> >wrote:
>> >>
>> >> I don't think we are talking about the same thing. You can certainly code 
>> >> an immutable object - just don't export any methods that mutate the 
>> >> object, nor export ANY fields.
>> >
>> >Correct, we're talking about different things. The question is not
>> >whether you can write an immutable object (yes you can), it is whether
>> >there is a way to enforce the immutability of the receiver of a
>> >method.
>> >
>> >If the method is exported and if the receiver contains pointers, there
>> >can be no guarantee that the method will not modify values reachable
>> >from the copy of the receiver.
>> >
>> >>
>> >>
>> >>
>> >>
>> >> -Original Message-
>> >> >From: burak serdar 
>> >> >Sent: Nov 21, 2019 11:09 AM
>> >> >To: Robert Engels 
>> >> >Cc: advanderv...@gmail.com, golang-nuts 
>> >> >Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >> >
>> >> >On Thu, Nov 21, 2019 at 10:05 AM Robert Engels  
>> >> >wrote:
>> >> >>
>> >> >> To clarify - the author of the package enforces immutability. With 
>> >> >> Go’s design this can be a simple comment on the field. The package 
>> >> >> shouldn’t be that large where this doesn’t work.
>> >> >
>> >> >The original problem remains: there is no way to enforce an immutable 
>> >> >receiver.
>> >> >
>> >> >>
>> >> >> > On Nov 21, 2019, at 10:58 AM, Robert Engels  
>> >> >> > wrote:
>> >> >> >
>> >> >> > 
>> >> >> > Correct, but if the receiver method is mutating it, then it is not 
>> >> >> > an immutable object.
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > -Original Message-
>> >> >> >> From: burak serdar 
>> >> >> >> Sent: Nov 21, 2019 10:53 AM
>> >> >> >> To: Robert Engels 
>> >> >> >> Cc: advanderv...@gmail.com, golang-nuts 
>> >> >> >> 
>> >> >> >> Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >> >> >>
>> >> >> >>> On Thu, Nov 21, 2019 at 9:49 AM Robert Engels 
>> >> >> >>>  wrote:
>> >> >> >>>
>> >> >> >>> They can't unless the instance field i

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

I'm sorry, but isn't the way you enforce immutability is that you don't code 
mutating exported methods? The author controls the package code. Immutability 
is usually only a concern outside the package, and that is clearly supported.



-Original Message-
>From: burak serdar 
>Sent: Nov 21, 2019 11:34 AM
>To: Robert Engels 
>Cc: advanderv...@gmail.com, golang-nuts 
>Subject: Re: [go-nuts] Enforce immutability through static analysis
>
>On Thu, Nov 21, 2019 at 10:25 AM Robert Engels  wrote:
>>
>> I don't think we are talking about the same thing. You can certainly code an 
>> immutable object - just don't export any methods that mutate the object, nor 
>> export ANY fields.
>
>Correct, we're talking about different things. The question is not
>whether you can write an immutable object (yes you can), it is whether
>there is a way to enforce the immutability of the receiver of a
>method.
>
>If the method is exported and if the receiver contains pointers, there
>can be no guarantee that the method will not modify values reachable
>from the copy of the receiver.
>
>>
>>
>>
>>
>> -Original Message-
>> >From: burak serdar 
>> >Sent: Nov 21, 2019 11:09 AM
>> >To: Robert Engels 
>> >Cc: advanderv...@gmail.com, golang-nuts 
>> >Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >
>> >On Thu, Nov 21, 2019 at 10:05 AM Robert Engels  
>> >wrote:
>> >>
>> >> To clarify - the author of the package enforces immutability. With Go’s 
>> >> design this can be a simple comment on the field. The package shouldn’t 
>> >> be that large where this doesn’t work.
>> >
>> >The original problem remains: there is no way to enforce an immutable 
>> >receiver.
>> >
>> >>
>> >> > On Nov 21, 2019, at 10:58 AM, Robert Engels  
>> >> > wrote:
>> >> >
>> >> > 
>> >> > Correct, but if the receiver method is mutating it, then it is not an 
>> >> > immutable object.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > -Original Message-
>> >> >> From: burak serdar 
>> >> >> Sent: Nov 21, 2019 10:53 AM
>> >> >> To: Robert Engels 
>> >> >> Cc: advanderv...@gmail.com, golang-nuts 
>> >> >> Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >> >>
>> >> >>> On Thu, Nov 21, 2019 at 9:49 AM Robert Engels  
>> >> >>> wrote:
>> >> >>>
>> >> >>> They can't unless the instance field is exported. Just hide it via 
>> >> >>> encapsulation with accessors.
>> >> >>
>> >> >> Can't do that with a receiver. All methods of a type are in the same
>> >> >> package as the type, so all fields are visible to the receiver.
>> >> >>
>> >> >>
>> >> >>>
>> >> >>> -Original Message-
>> >> >>> From: advanderv...@gmail.com
>> >> >>> Sent: Nov 21, 2019 10:15 AM
>> >> >>> To: golang-nuts
>> >> >>> Subject: [go-nuts] Enforce immutability through static analysis
>> >> >>>
>> >> >>> Dear Gophers!
>> >> >>>
>> >> >>> I was wonder if it possible to force immutability on the method 
>> >> >>> receiver? I know Go doesn't support immutable types and that it is 
>> >> >>> possible to pass the receiver by value but if the receiver struct has 
>> >> >>> a field with a pointer type the method may still manipulate it:
>> >> >>>
>> >> >>> type Counter struct {
>> >> >>> n *int
>> >> >>> }
>> >> >>>
>> >> >>> func (c Counter) Render() string {
>> >> >>> *c.n += 1
>> >> >>> return strconv.Itoa(*c.n)
>> >> >>> }
>> >> >>>
>> >> >>> I would like to force (or hint) the the user in writing interface{ 
>> >> >>> Render() string } implementations that don't manipulate the method 
>> >> >>> receiver. So that they can be considered 'pure' in the functional 
>> >> >>> sense of the word and can b

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

I don't think we are talking about the same thing. You can certainly code an 
immutable object - just don't export any methods that mutate the object, nor 
export ANY fields.




-Original Message-
>From: burak serdar 
>Sent: Nov 21, 2019 11:09 AM
>To: Robert Engels 
>Cc: advanderv...@gmail.com, golang-nuts 
>Subject: Re: [go-nuts] Enforce immutability through static analysis
>
>On Thu, Nov 21, 2019 at 10:05 AM Robert Engels  wrote:
>>
>> To clarify - the author of the package enforces immutability. With Go’s 
>> design this can be a simple comment on the field. The package shouldn’t be 
>> that large where this doesn’t work.
>
>The original problem remains: there is no way to enforce an immutable receiver.
>
>>
>> > On Nov 21, 2019, at 10:58 AM, Robert Engels  wrote:
>> >
>> > 
>> > Correct, but if the receiver method is mutating it, then it is not an 
>> > immutable object.
>> >
>> >
>> >
>> >
>> > -Original Message-
>> >> From: burak serdar 
>> >> Sent: Nov 21, 2019 10:53 AM
>> >> To: Robert Engels 
>> >> Cc: advanderv...@gmail.com, golang-nuts 
>> >> Subject: Re: [go-nuts] Enforce immutability through static analysis
>> >>
>> >>> On Thu, Nov 21, 2019 at 9:49 AM Robert Engels  
>> >>> wrote:
>> >>>
>> >>> They can't unless the instance field is exported. Just hide it via 
>> >>> encapsulation with accessors.
>> >>
>> >> Can't do that with a receiver. All methods of a type are in the same
>> >> package as the type, so all fields are visible to the receiver.
>> >>
>> >>
>> >>>
>> >>> -Original Message-
>> >>> From: advanderv...@gmail.com
>> >>> Sent: Nov 21, 2019 10:15 AM
>> >>> To: golang-nuts
>> >>> Subject: [go-nuts] Enforce immutability through static analysis
>> >>>
>> >>> Dear Gophers!
>> >>>
>> >>> I was wonder if it possible to force immutability on the method 
>> >>> receiver? I know Go doesn't support immutable types and that it is 
>> >>> possible to pass the receiver by value but if the receiver struct has a 
>> >>> field with a pointer type the method may still manipulate it:
>> >>>
>> >>> type Counter struct {
>> >>> n *int
>> >>> }
>> >>>
>> >>> func (c Counter) Render() string {
>> >>> *c.n += 1
>> >>> return strconv.Itoa(*c.n)
>> >>> }
>> >>>
>> >>> I would like to force (or hint) the the user in writing interface{ 
>> >>> Render() string } implementations that don't manipulate the method 
>> >>> receiver. So that they can be considered 'pure' in the functional sense 
>> >>> of the word and can be called repeatedly without side effects. I would 
>> >>> like the user to be able to define implementations of interface{ 
>> >>> Render() string }such that I can safely call the method and use the 
>> >>> returned string to write a http.Reponse without it changing between 
>> >>> requests.
>> >>>
>> >>> I control the way in which Render is called and I am open to crazy 
>> >>> answers such as:
>> >>>
>> >>> - Maybe it is possible to use reflect to "switch" out the value receiver 
>> >>> for a temporary value which is discarded after every call?
>> >>> - Maybe i can use static code analysis to warn the user? How feasible is 
>> >>> it to prevent all cases of this happening with just static code 
>> >>> analysis? can this be done at runtime?
>> >>> - I could instead ask the user to provide a factory function that init 
>> >>> new Counters but maybe very inefficient if the structs are very large 
>> >>> (or have many nested structs)?
>> >>>
>> >>> Or maybe there is some possibility that I'm missing?
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Ad
>> >>>
>> >>>
>> >>> --
>> >>> You received this message because you are subscribed to the Google 
>> >>> Groups "golang-nuts" group.
>> >>> To unsubscribe from this group and stop receiving emails from it, send 
>

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

To clarify - the author of the package enforces immutability. With Go’s design 
this can be a simple comment on the field. The package shouldn’t be that large 
where this doesn’t work. 

> On Nov 21, 2019, at 10:58 AM, Robert Engels  wrote:
> 
> 
> Correct, but if the receiver method is mutating it, then it is not an 
> immutable object.
> 
> 
> 
> 
> -Original Message-
>> From: burak serdar 
>> Sent: Nov 21, 2019 10:53 AM
>> To: Robert Engels 
>> Cc: advanderv...@gmail.com, golang-nuts 
>> Subject: Re: [go-nuts] Enforce immutability through static analysis
>> 
>>> On Thu, Nov 21, 2019 at 9:49 AM Robert Engels  wrote:
>>> 
>>> They can't unless the instance field is exported. Just hide it via 
>>> encapsulation with accessors.
>> 
>> Can't do that with a receiver. All methods of a type are in the same
>> package as the type, so all fields are visible to the receiver.
>> 
>> 
>>> 
>>> -Original Message-
>>> From: advanderv...@gmail.com
>>> Sent: Nov 21, 2019 10:15 AM
>>> To: golang-nuts
>>> Subject: [go-nuts] Enforce immutability through static analysis
>>> 
>>> Dear Gophers!
>>> 
>>> I was wonder if it possible to force immutability on the method receiver? I 
>>> know Go doesn't support immutable types and that it is possible to pass the 
>>> receiver by value but if the receiver struct has a field with a pointer 
>>> type the method may still manipulate it:
>>> 
>>> type Counter struct {
>>> n *int
>>> }
>>> 
>>> func (c Counter) Render() string {
>>> *c.n += 1
>>> return strconv.Itoa(*c.n)
>>> }
>>> 
>>> I would like to force (or hint) the the user in writing interface{ Render() 
>>> string } implementations that don't manipulate the method receiver. So that 
>>> they can be considered 'pure' in the functional sense of the word and can 
>>> be called repeatedly without side effects. I would like the user to be able 
>>> to define implementations of interface{ Render() string }such that I can 
>>> safely call the method and use the returned string to write a http.Reponse 
>>> without it changing between requests.
>>> 
>>> I control the way in which Render is called and I am open to crazy answers 
>>> such as:
>>> 
>>> - Maybe it is possible to use reflect to "switch" out the value receiver 
>>> for a temporary value which is discarded after every call?
>>> - Maybe i can use static code analysis to warn the user? How feasible is it 
>>> to prevent all cases of this happening with just static code analysis? can 
>>> this be done at runtime?
>>> - I could instead ask the user to provide a factory function that init new 
>>> Counters but maybe very inefficient if the structs are very large (or have 
>>> many nested structs)?
>>> 
>>> Or maybe there is some possibility that I'm missing?
>>> 
>>> Cheers,
>>> 
>>> Ad
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/7ee35405-fef4-415b-ae5d-95322b4065aa%40googlegroups.com.
>>> 
>>> 
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/1622995561.1365.1574354931169%40wamui-scooby.atl.sa.earthlink.net.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/2080138990.1391.1574355466613%40wamui-scooby.atl.sa.earthlink.net.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/F9A0E915-48EB-4177-9DA6-C0AC0B1C977F%40ix.netcom.com.

Re: [go-nuts] Enforce immutability through static analysis

[Robert Engels]

Correct, but if the receiver method is mutating it, then it is not an immutable 
object.




-Original Message-
>From: burak serdar 
>Sent: Nov 21, 2019 10:53 AM
>To: Robert Engels 
>Cc: advanderv...@gmail.com, golang-nuts 
>Subject: Re: [go-nuts] Enforce immutability through static analysis
>
>On Thu, Nov 21, 2019 at 9:49 AM Robert Engels  wrote:
>>
>> They can't unless the instance field is exported. Just hide it via 
>> encapsulation with accessors.
>
>Can't do that with a receiver. All methods of a type are in the same
>package as the type, so all fields are visible to the receiver.
>
>
>>
>> -Original Message-
>> From: advanderv...@gmail.com
>> Sent: Nov 21, 2019 10:15 AM
>> To: golang-nuts
>> Subject: [go-nuts] Enforce immutability through static analysis
>>
>> Dear Gophers!
>>
>> I was wonder if it possible to force immutability on the method receiver? I 
>> know Go doesn't support immutable types and that it is possible to pass the 
>> receiver by value but if the receiver struct has a field with a pointer type 
>> the method may still manipulate it:
>>
>> type Counter struct {
>>  n *int
>> }
>>
>> func (c Counter) Render() string {
>>  *c.n += 1
>>  return strconv.Itoa(*c.n)
>> }
>>
>> I would like to force (or hint) the the user in writing interface{ Render() 
>> string } implementations that don't manipulate the method receiver. So that 
>> they can be considered 'pure' in the functional sense of the word and can be 
>> called repeatedly without side effects. I would like the user to be able to 
>> define implementations of interface{ Render() string }such that I can safely 
>> call the method and use the returned string to write a http.Reponse without 
>> it changing between requests.
>>
>> I control the way in which Render is called and I am open to crazy answers 
>> such as:
>>
>> - Maybe it is possible to use reflect to "switch" out the value receiver for 
>> a temporary value which is discarded after every call?
>> - Maybe i can use static code analysis to warn the user? How feasible is it 
>> to prevent all cases of this happening with just static code analysis? can 
>> this be done at runtime?
>> - I could instead ask the user to provide a factory function that init new 
>> Counters but maybe very inefficient if the structs are very large (or have 
>> many nested structs)?
>>
>> Or maybe there is some possibility that I'm missing?
>>
>> Cheers,
>>
>> Ad
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/7ee35405-fef4-415b-ae5d-95322b4065aa%40googlegroups.com.
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/1622995561.1365.1574354931169%40wamui-scooby.atl.sa.earthlink.net.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2080138990.1391.1574355466613%40wamui-scooby.atl.sa.earthlink.net.