[go-nuts] expected number of pprof 'total samples'

2021-04-29 Thread Xiangdong Ji
Hi I wonder if the expected number of pprof 'total samples' could be calculated by: duration * sample_rate (100HZ) * CPU usage (by 'top') say if my program's CPU usage reported by 'top' is 200%, and sampling period is 30 seconds, could I expect the total samples of pprof is

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Ian Lance Taylor
On Thu, Apr 29, 2021 at 12:05 PM Øyvind Teig wrote: > > The example here is a server with N clients where it is essential that none > of clients will starve and none jam the server. I have needed to do this > coding several times. Go has random select which in theory may mean starving > and

Re: [go-nuts] TotalAlloc dropped in runtime/metrics

2021-04-29 Thread 'Michael Knyszek' via golang-nuts
Hi Marco, https://go-review.googlesource.com/c/go/+/312431 just landed and should resolve the issue for Go 1.17. On Tue, Apr 20, 2021 at 3:28 PM Michael Knyszek wrote: > Oh, actually, you can compute Mallocs and Frees from allocs-by-size and > frees-by-size (summing the total # of samples). You

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread roger peppe
On Thu, 29 Apr 2021, 20:05 Øyvind Teig, wrote: > torsdag 29. april 2021 kl. 20:22:32 UTC+2 skrev rog: > >> I agree with Axel's take here. It seems, Øyvind, that you are concerned >> more with principle than practice here. Can you give an example of a real >> world case where you think that this

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread 'Axel Wagner' via golang-nuts
On Thu, Apr 29, 2021 at 10:47 PM Øyvind Teig wrote: > >1. I'm sorry I didn't follow up on your answer where you had got to >the length of spelling out some code right before my eyes. I got lost in >the other points (I actually started a response..) >2. I trust you >3. But to

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
1. I'm sorry I didn't follow up on your answer where you had got to the length of spelling out some code right before my eyes. I got lost in the other points (I actually started a response..) 2. I trust you 3. But to do more than trusting, understanding would be much better.

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Robert Engels
I already gave you the solution to this. Trust me it works. I did HFT for 7+ years - the algo is well known. > On Apr 29, 2021, at 2:05 PM, Øyvind Teig wrote: > >  >> torsdag 29. april 2021 kl. 20:22:32 UTC+2 skrev rog: > >> I agree with Axel's take here. It seems, Øyvind, that you are

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
torsdag 29. april 2021 kl. 20:22:32 UTC+2 skrev rog: > I agree with Axel's take here. It seems, Øyvind, that you are concerned > more with principle than practice here. Can you give an example of a real > world case where you think that this might actually matter? > Thanks, yes. I have written

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread roger peppe
I agree with Axel's take here. It seems, Øyvind, that you are concerned more with principle than practice here. Can you give an example of a real world case where you think that this might actually matter? On Thu, 29 Apr 2021, 15:44 'Axel Wagner' via golang-nuts, < golang-nuts@googlegroups.com>

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread 'Axel Wagner' via golang-nuts
FWIW, maybe this helps: Assume a read happened from lowPriority, even though highPriority was ready to read as well. That's, AIUI, the outcome you are concerned about. In that situation, how would you know that highPriority was ready to read as well? On Thu, Apr 29, 2021 at 4:39 PM Axel Wagner

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread 'Axel Wagner' via golang-nuts
On Thu, Apr 29, 2021 at 3:54 PM Øyvind Teig wrote: > They could still both have become ready (not in the same "cycle") between > the two selects. Even if that probability is low, it would need knowledge > like yours to show that this may in fact be zero. There could be a > descheduling in

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread robert engels
To clarify, you might never process a low item if a high is always available - but I believe that is what the OP is requesting. It is also fairly trivial to add fairness to this ti always process a low if available every N cycles. > On Apr 29, 2021, at 9:31 AM, robert engels wrote: > > It is

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread robert engels
It is not. Because the low items will be read if there are no highs available. If the low isn’t read, the producer on the low will block. Or you run out of memory if you cannot process the events fast enough - which is also easily handled by bounding the queue - but then you may need to drop

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread robert engels
It needs one other fix, and that is to have a check of the queue and poll high, in case the queue has elements when the method is entered. But if you implement the queue using a buffered channel it is trivial to change the step #1 to handle this (the low and queue channels are mimic the high

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Ian Davis
On Thu, 29 Apr 2021, at 3:09 PM, robert engels wrote: > I will give you the pseudo code: > > { // step #1 do select on both >select high >select low > } > > if high read: >return high > else: >// we read a low so do a high poll >{ >select high: >default: >

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Jan Mercl
On Thu, Apr 29, 2021 at 3:54 PM Øyvind Teig wrote: > Even if that probability is low, it would need knowledge like yours to show > that this may in fact be zero. There could be a descheduling in between, one > of those in my opinion, not relevant arguments. The upper limit of 4c happening is

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread robert engels
I will give you the pseudo code: { // step #1 do select on both select high select low } if high read: return high else: // we read a low so do a high poll { select high: default: } if high read: enqueue low and return high else: if queue empty:

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
They could still both have become ready (not in the same "cycle") between the two selects. Even if that probability is low, it would need knowledge like yours to show that this may in fact be zero. There could be a descheduling in between, one of those in my opinion, not relevant arguments.

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Jan Mercl
On Thu, Apr 29, 2021 at 3:23 PM Øyvind Teig wrote: > 4c is not "correct" as I want it. In the pri select case, if more than one is > ready, then they shall not be randomly chosen. Never. They should be selected > according to priority. That's not what 4c says. Instead of "more than one ready"

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
axel, it's difficult to try to explain in words how a select behaves. On that we can agree. For me, select cases can be channels, timeouts, external events etc. If a task is in a select (none triggered yet) and there is an interrupt that detects the select entry, it's ticked as triggered, the

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
I sent that to the wrong entry! Is it possible that you could make a code example? torsdag 29. april 2021 kl. 15:20:40 UTC+2 skrev ren...@ix.netcom.com: > It is trivial to do a priority select. Check the high priority channel > again after getting a low priority value. If the high priority is

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread 'Axel Wagner' via golang-nuts
My argument is that you can't distinguish the case "neither was ready and highPriority and lowPriority became ready simultaneously and lowPriority was selected" from "highPriority became ready after lowPriority" (as long as there is the same code in both highPriority cases). Because there is no

Re: [go-nuts] HTTP request reading

2021-04-29 Thread K. Alex Mills
Partial responses inline, HTH. On Thu, Apr 29, 2021, 6:09 AM Amit Saha wrote: > Hi all, when an incoming request comes in, does the ListenAndServe() > function read the first line (as explained in > https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages) to figure > out whether there is a

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
Jan, I agree on your list, it's better set up than mine. But you can add a point after (4.) though, call it (second.pre), dealing with what might have happened after the first select decided that none were taken until it evaluates the precondition to the second select. 4c is not "correct" as

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Robert Engels
It is trivial to do a priority select. Check the high priority channel again after getting a low priority value. If the high priority is available enqueue the low value and handle the high. It is also trivial to “delay” the handling to allow more time for high events to arrive if dealing with

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
I agree with you. However, with the first select in the example we have a "first" and with the second we have an "after" which might be considered race condition(s9 for the purpose of seeing the example of what I was after. Also, scheduling arguments cannot be used here: like "we know that

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Jan Mercl
On Thu, Apr 29, 2021 at 2:26 PM Øyvind Teig wrote: > Your suggestion would in fact do pri select in the special case 1. below: > > Poll highPri first (take it if it's ready), if highPri not ready then take > lowPri (provided highPri has not become ready since the first poll) > However, if

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread 'Axel Wagner' via golang-nuts
On Thu, Apr 29, 2021 at 2:26 PM Øyvind Teig wrote: > Interesting! > > Your suggestion would in fact do pri select in the special case 1. below: > >1. Poll highPri first (take it if it's ready), if highPri not ready >then take lowPri (provided highPri has not become ready since the first

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
If not, pick randomly -> If so, pick randomly torsdag 29. april 2021 kl. 14:26:15 UTC+2 skrev Øyvind Teig: > Interesting! > > Your suggestion would in fact do pri select in the special case 1. below: > >1. Poll highPri first (take it if it's ready), if highPri not ready >then take

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
Interesting! Your suggestion would in fact do pri select in the special case 1. below: 1. Poll highPri first (take it if it's ready), if highPri not ready then take lowPri (provided highPri has not become ready since the first poll) 2. However, if highPri has become ready between the

Re: [go-nuts] Re: How to do vdso calls in my own code?

2021-04-29 Thread Robert Engels
The point is that if you are timing “so much” that a 1-2 % overhead (so you must have tons of very small operations) matter to throughput - sampling will still capture enough that a representative event will be recorded. Still, optimizing time.Now() is always a good idea but I am guessing much

[go-nuts] HTTP request reading

2021-04-29 Thread Amit Saha
Hi all, when an incoming request comes in, does the ListenAndServe() function read the first line (as explained in https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages) to figure out whether there is a handler registered or not for the path and then simply hands it over to the handler (if

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Jan Mercl
On Thu, Apr 29, 2021 at 11:24 AM Øyvind Teig wrote: > This is not solved with a default clause, which transforms the selective > choice waiting for some event to happen into busy polling. It's nice yo have > some times, but that is something orthogonal to pri/ordered. Not sure if I would call

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
Thanks! A construct like "pri select" or "alt select" or "ordered select", so it's not a "wrt", it's a *tagging* of select changing ets sematics from being nondeterminstic to deterministic. CSP has both: external choice (nondeterminsitic), internal choice (deterministic). The occam

Re: [go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Jan Mercl
On Thu, Apr 29, 2021 at 10:52 AM Øyvind Teig wrote: > I know from some years ago that go did not have any priority or ordered > select construct [1]. Not sure what does "ordered" wrt select mean, but the default clause of the select statement provides exactly that - a priority mechanism: first

[go-nuts] Still "missing" priority or ordered select in go?

2021-04-29 Thread Øyvind Teig
I know from some years ago that go did not have any priority or ordered select construct [1]. The idea is that select always is a nondeterministic choice operator, if this is still so. Implemented with a random, I assume. Programming user defined "fair" algorithms or patterns is then not

Re: [go-nuts] Re: Multi-word Project Name

2021-04-29 Thread Brian Candler
Sure. In the OP's given path "github.com/username/sherlock-server", if it provides either "package sherlock" or "package server" it's not too bad. Thanks for pointing me to goimports. That does indeed rewrite the example as import ( "fmt" foo "example.com/my/module/with-a-very-weird-name" )

Re: [go-nuts] Re: Multi-word Project Name

2021-04-29 Thread 'Axel Wagner' via golang-nuts
Note that goimports theses days adds the package name explicitly to the import if the package name is not identical to the last component of the import path. So on the readability front, the cost seem pretty low to me. It does have a cost on the writing side, as goimports might not find the

Re: [go-nuts] Re: Multi-word Project Name

2021-04-29 Thread Brian Candler
P.S. I don't really recommend this, since you can't tell by inspection of main.go which import provides "foo.Greeting". And I'm struggling to think of any real-world examples where I've seen this used. -- You received this message because you are subscribed to the Google Groups "golang-nuts"

Re: [go-nuts] Re: Multi-word Project Name

2021-04-29 Thread Brian Candler
> Guess by "even if it wasn't" you mean "even if it wasn't top level and was exporting functionality" I think he means in general, you can do this: // go.mod says "module github.com/my/module/with-a-very-weird-name" package foo ... and when you import it, you still get a package "foo". That's

Re: [go-nuts] Re: How to do vdso calls in my own code?

2021-04-29 Thread Amnon
Why not use the TSC timer for your timings, and convert TSC cycle times into durations during post-run analysis? On Thursday, 29 April 2021 at 04:04:58 UTC+1 Pure White wrote: > Hello, > We are doing heavily tracing, and for each func we need to get the time, > so that’s why I’d like to