Re: [go-nuts] AI-first?

[Nigel Tao]

On Fri, Jun 2, 2017 at 7:17 AM, Ian Lance Taylor  wrote:
> The golang-nuts mailing list is a good place to talk about Go-first
> (or Go-second or Go-last).  It's not a good place to talk about AI.
> Thanks.

In case there is any confusion, this list is about Go the programming
language. There's also interesting news about Artificial Intelligence
and Go the board game, but as Ian said, this list is not the right
place to discuss that, unless of course it also involves Go the
programming language.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Dereferencing pointers, function evaluation and tuple assignment

[Will Hawkins]

On Fri, Jun 2, 2017 at 1:03 AM, Ian Lance Taylor  wrote:
> On Thu, Jun 1, 2017 at 9:52 PM, Will Hawkins  wrote:
>> On Fri, Jun 2, 2017 at 12:41 AM, Ian Lance Taylor  wrote:
>>>
>>> Unlike function calls, the language spec doesn't anything about when a
>>> dereference operation is executed.  And as you can see the current
>>> compiler does not do the dereference immediately after the function
>>> call.
>>
>> This is the bit of information that I was hoping you'd have. It seems
>> like an odd omission from the spec. On one hand, it clearly allows the
>> compiler to perform optimizations that otherwise might not be
>> possible. On the other, though, it would seem to introduce the
>> possibility of data races or, as in the case here, unexpected results.
>> Just my naive opinion. If you had thoughts about it, I'd love to hear
>> your expert opinion.
>
> I would say that technically it does not introduce the possibility of
> data races.  The only way to avoid a data race in Go is to use a
> channel operation or an explicit lock, such as a sync.Mutex, which
> requires a function call.  There is a specified order of evaluation
> for both channel operations and function calls, so there is no
> possible confusion about whether there is a data race there.  If your
> program can have a data race due to two goroutines doing a pointer
> indirection simultaneously, then your program has a data race
> regardless of the order of execution.

You responded to the question that I *meant* to ask, and I appreciate
it. I was sloppily referring to the possibility that this particular
omission allows for a style of program that is not the, shall we say,
clearest. And that seems to be exactly the second reason that you
state below -- specifying this with any particularity would imply that
it is "okay" to have programs like this.

Thanks again for your thorough and complete response. It's an honor to
talk to someone who has such a record of work in really neat,
technical open source software.

Will

>
> It is true that you can have unexpected results, but only if you are
> using a style of programming that is rather prone to
> misunderstandings.  So it seems hard to get too worked up by it.
>
> Anyhow, to answer your real question, at various times we did discuss
> fully specifying the order of evaluation of all operations.  But we've
> never decided to do it for the reason you mention: it puts significant
> constraints on the compiler.  That, combined with the fact that the
> kinds of programs where the order of evaluation makes a difference are
> programs that are already confusing, makes it seem not worth
> specifying in the language.
>
>
>> I sincerely hope that what I asked was not a stupid question. It gave
>> me lots of trouble and I worked through the problem with several
>> others and we could not figure out the cause of the discrepancy.
>
> No worries, it is not at all a stupid question.
>
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Dereferencing pointers, function evaluation and tuple assignment

[Will Hawkins]

On Fri, Jun 2, 2017 at 12:41 AM, Ian Lance Taylor  wrote:
> On Thu, Jun 1, 2017 at 6:40 PM, Will Hawkins  wrote:
>> Hello awesome community!
>>
>> I am so honored to be a part of the great community of gophers around the
>> world. Thank you to everyone who makes it so inviting to everyone who wants
>> to participate.
>
> Thanks for the nice intro.

Thank you for your quick and very helpful reply, Mr. Taylor. It turns
out I read a previous message of yours a few days ago on an unrelated
list (about poison malloc in gcc) so it's quite the coincidence!

>
>
>> I am writing today to ask about some puzzling behavior with respect to
>> dereferencing pointers, function evaluation and tuple assignment. I have the
>> following code snippet:
>>
>> type T struct {
>> i int
>> }
>>
>> func (t *T) sideEffectPtr() *int {
>> t.i += 4
>> return 
>> }
>>
>> func derefIntPtr(i *int) int {
>> return *i
>> }
>>
>> func main() {
>> var d = T{i: 200}
>> var e = T{i: 200}
>> a, b := derefIntPtr(d.sideEffectPtr()), derefIntPtr(d.sideEffectPtr())
>> aa, bb := *e.sideEffectPtr(), *e.sideEffectPtr()
>> fmt.Println(a, b)
>> fmt.Println(aa, bb)
>> }
>>
>> You can see the code in action on the playground here:
>> https://play.golang.org/p/wGsvYWXEqf
>>
>> The output is surprising, to say the least:
>> 204 208
>> 208 208
>>
>>
>> From what I understand, the right hand side of a tuple assignment is
>> completely evaluated before the assignment happens.
>
> That is true, but I don't think it's relevant here.  That tells us
> that the two instances of `*e.sideEffectPtr()` are evaluated before
> either value is assigned to aa or bb.  But it doesn't tell us anything
> about the exact sequence of steps used to evaluate those two
> instances.

Thank you for that clarification.

>
>
>> I also think that I
>> understand that for evaluations not covered specifically by the language
>> specification, the particular order of the evaluation is not defined. In
>> other words, in
>>
>> aa, bb := *e.sideEffectPtr(), *e.sideEffectPtr()
>>
>> the calls to sideEffectPtr() may happen in any order.
>
> That turns out not to be the case.  The "Order of Evaluations" section
> of the spec says that in an assignment statement all function calls
> are evaluated in lexical left-to-right order.
>

You are, obviously, absolutely correct. I am embarrassed to say that I
mistranslated my understanding of the spec into my message. Thank you
for making the correction.

>
>> However, I would imagine that, no matter what the order, the dereference
>> operation would happen immediately after the first function call and before
>> the second. That would lead me to believe that one output would be 204 and
>> the other would be 208.
>
> Unlike function calls, the language spec doesn't anything about when a
> dereference operation is executed.  And as you can see the current
> compiler does not do the dereference immediately after the function
> call.

This is the bit of information that I was hoping you'd have. It seems
like an odd omission from the spec. On one hand, it clearly allows the
compiler to perform optimizations that otherwise might not be
possible. On the other, though, it would seem to introduce the
possibility of data races or, as in the case here, unexpected results.
Just my naive opinion. If you had thoughts about it, I'd love to hear
your expert opinion.

I sincerely hope that what I asked was not a stupid question. It gave
me lots of trouble and I worked through the problem with several
others and we could not figure out the cause of the discrepancy.

Thanks again for your response!

Will

>
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Dereferencing pointers, function evaluation and tuple assignment

[Ian Lance Taylor]

On Thu, Jun 1, 2017 at 6:40 PM, Will Hawkins  wrote:
> Hello awesome community!
>
> I am so honored to be a part of the great community of gophers around the
> world. Thank you to everyone who makes it so inviting to everyone who wants
> to participate.

Thanks for the nice intro.


> I am writing today to ask about some puzzling behavior with respect to
> dereferencing pointers, function evaluation and tuple assignment. I have the
> following code snippet:
>
> type T struct {
> i int
> }
>
> func (t *T) sideEffectPtr() *int {
> t.i += 4
> return 
> }
>
> func derefIntPtr(i *int) int {
> return *i
> }
>
> func main() {
> var d = T{i: 200}
> var e = T{i: 200}
> a, b := derefIntPtr(d.sideEffectPtr()), derefIntPtr(d.sideEffectPtr())
> aa, bb := *e.sideEffectPtr(), *e.sideEffectPtr()
> fmt.Println(a, b)
> fmt.Println(aa, bb)
> }
>
> You can see the code in action on the playground here:
> https://play.golang.org/p/wGsvYWXEqf
>
> The output is surprising, to say the least:
> 204 208
> 208 208
>
>
> From what I understand, the right hand side of a tuple assignment is
> completely evaluated before the assignment happens.

That is true, but I don't think it's relevant here.  That tells us
that the two instances of `*e.sideEffectPtr()` are evaluated before
either value is assigned to aa or bb.  But it doesn't tell us anything
about the exact sequence of steps used to evaluate those two
instances.


> I also think that I
> understand that for evaluations not covered specifically by the language
> specification, the particular order of the evaluation is not defined. In
> other words, in
>
> aa, bb := *e.sideEffectPtr(), *e.sideEffectPtr()
>
> the calls to sideEffectPtr() may happen in any order.

That turns out not to be the case.  The "Order of Evaluations" section
of the spec says that in an assignment statement all function calls
are evaluated in lexical left-to-right order.


> However, I would imagine that, no matter what the order, the dereference
> operation would happen immediately after the first function call and before
> the second. That would lead me to believe that one output would be 204 and
> the other would be 208.

Unlike function calls, the language spec doesn't anything about when a
dereference operation is executed.  And as you can see the current
compiler does not do the dereference immediately after the function
call.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] how to build websockert connection thru http proxy

[273593...@qq.com]

question is as subject, any sample would be appreciated

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: "cannot allocate memory" problem

[Dmitry Mishin]

I was wrong about the number of folder read workers. Adding more workers 
makes the error happen sooner.

On Thursday, June 1, 2017 at 6:24:48 PM UTC-7, Dmitry Mishin wrote:
>
> What I'm wondering about is the time it takes to get the error - very 
>> close to 10 minutes all the time. Not even dependent on the number of 
>> workers (I have a setting for that and trying with 1-5 workers)
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: "cannot allocate memory" problem

[Dmitry Mishin]

The system has 1GB of swap, and I just tried enabling:

sysctl -w vm.overcommit_memory=1
sysctl -w vm.swappiness=1

according to 
https://stackoverflow.com/questions/35025338/cannot-allocate-memory-error, 
with no effect.

There's 64GB of RAM, which is not being used at the time of error:

top - 16:47:14 up 2 days,  4:35,  4 users,  load average: 1.43, 1.35, 1.12
Tasks: 32606 total,   1 running, 531 sleeping,   0 stopped, 32074 zombie
Cpu(s):  0.7%us, 14.0%sy,  0.0%ni, 80.6%id,  3.3%wa,  0.0%hi,  1.4%si,  0.0%
st
Mem:  66067872k total, 43270952k used, 22796920k free, 5684k buffers
Swap:  1023996k total,  332k used,  1023664k free, 26004700k cached


  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
 1401 root  20   0  898m  26m 2936 S 93.6  0.0   9:30.36 pdm
 6494 root  20   0 38280  25m  880 R 20.1  0.0   1:20.34 top
 6145 root  20   0  112m 5468 4428 S  0.0  0.0   0:00.05 sshd
29324 root  20   0  112m 5092 4056 S  0.0  0.0   0:00.60 sshd
 2720 root  20   0  114m 3284  616 S  0.3  0.0   0:43.29 sshd
 2377 nobody20   0  152m 3148  668 S  0.0  0.0   0:06.24 gmond
 9290 postfix   20   0 79868 2852 1996 S  0.0  0.0   0:00.01 pickup
29336 root  20   0  106m 1916 1432 S  0.0  0.0   0:00.03 bash
 6155 root  20   0  106m 1888 1428 S  0.0  0.0   0:00.02 bash
 2575 root  20   0  379m 1880  684 S  0.0  0.0   0:00.89 automount
 2240 haldaemo  20   0 38224 1744  632 S  0.0  0.0   0:01.58 hald
30608 root  20   0  112m 1672  636 S  0.0  0.0   0:00.24 sshd
 2724 root  20   0  106m 1416  912 S  0.0  0.0   0:00.31 bash
 2467 postfix   20   0 80036 1404  496 S  0.0  0.0   0:00.18 qmgr
30634 root  20   0  106m 1376  868 S  0.0  0.0   0:00.13 bash
 2456 root  20   0 8 1352  460 S  0.0  0.0   0:00.74 master
 1911 root  20   0  245m 1300  588 S  0.0  0.0   0:00.27 rsyslogd
 2366 ntp   20   0 25440 1160  588 S  0.0  0.0   0:00.24 ntpd
 2323 nscd  20   0  877m 1148  644 S  0.0  0.0   0:02.14 nscd
 2357 root  20   0 90848 1052  392 S  0.0  0.0   0:00.01 sshd
 1628 root  18  -2 11260 1048  232 S  0.0  0.0   0:00.00 udevd
 2468 root  20   0  112m  980  392 S  0.0  0.0   0:00.40 crond

pprof is showing around 2MB memory used all the time.

I'm thinking the problem is somewhere here, since it goes away when I 
disable this part of the app:
https://github.com/sdsc/pdm/blob/3b5c7fcef24e9081f3bd0608efde1bdc10a65d17/lustre_backend.go#L73

Also I thought I'm creating too many goroutines too fast, and I just 
rewrote this part to use no goroutines and channels and return a simple 
slice, with no good effect:

https://github.com/sdsc/pdm/blob/master/lustre_backend.go#L73

What I'm wondering about is the time it takes to get the error - very close 
to 10 minutes all the time. Not even dependent on the number of workers (I 
have a setting for that and trying with 1-5 workers)

On Thursday, June 1, 2017 at 5:54:51 PM UTC-7, Dave Cheney wrote:
>
> Does the machine (vm, container, etc) you are running this application 
> have any swap configured.
>
> What I think is happening is the momentary spike in potential memory usage 
> during the fork / exec cycle is causing the linux memory system to refuse 
> to permit the fork. There are several open issues for this, search clone or 
> vfork on the github.com/golang/go project, but the short version is; add 
> swap.
>
> On Friday, 2 June 2017 10:41:10 UTC+10, Dmitry Mishin wrote:
>>
>> Hello all,
>>
>> Trying to fix the out of memory issue in my go app.
>>
>> The app is working fine with no memory usage increasing for 10-15 
>> minutes, then suddenly system starts giving "cannot allocate memory" error 
>> for any command, same in the app.
>>
>> I attached the trace and heap during the out of memory state
>>
>> The app is finding files and folders in lustre mount, submitting those as 
>> amqp messages, and copying the discovered files
>>
>> Thanks for your help!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: A spinning cube demo using Vulkan API and Go, the project status

[Erwin Driessens]

bravo!
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] Gomail v2: sending emails faster

[connelblaze]

Hey Alex, tried sending mail with gomail v2 having some ish.
here is my code:

func contact(w http.ResponseWriter, req *http.Request) {
   var s string
   var e string
   var m string
   pd := pageData{
  Title: "contact || CACCYE",
   }
   if req.Method == http.MethodPost {
  s = req.FormValue("subject")
  e = req.FormValue("mail")
  m = req.FormValue("message")

  pd.subject = s
  pd.email = e
  pd.msg = m

  cb := gomail.NewMessage()
  cb.SetHeader("From", e)
  cb.SetHeader("To", "his_m...@gmail.com", "her_m...@gmail.com")
  cb.SetHeader("Subject", s)
  cb.SetBody("text/plain", m)

  d := gomail.NewDialer("smtp.gmail.com", 587, "my_m...@gmail.com", 
"my_password")

  // Send the email to Bob, Cora and Dan.
  if err := d.DialAndSend(cb); err != nil {
 panic(err)
  }
   }

   err := tpl.ExecuteTemplate(w, "contact.gohtml", pd)
   if err != nil {
  log.Println(err)
   }
}


when i submit it outputs:
2017/06/01 19:28:48 http: panic serving [::1]:50340: dial tcp: i/o timeout
goroutine 6 [running]:
net/http.(*conn).serve.func1(0xc42008c6e0)
/usr/local/go/src/net/http/server.go:1721 +0xd0
panic(0x709800, 0xc4200165a0)
/usr/local/go/src/runtime/panic.go:489 +0x2cf
main.contact(0x8b6740, 0xc42013e2a0, 0xc42000ae00)
/home/connelblaze/go/src/github.com/connelevalsam/GoWebDev/001/src/sendmail/src/main/main.go:86
 
+0x5d3
net/http.HandlerFunc.ServeHTTP(0x769658, 0x8b6740, 0xc42013e2a0, 
0xc42000ae00)
/usr/local/go/src/net/http/server.go:1942 +0x44
net/http.(*ServeMux).ServeHTTP(0x8e38e0, 0x8b6740, 0xc42013e2a0, 
0xc42000ae00)
/usr/local/go/src/net/http/server.go:2238 +0x130
net/http.serverHandler.ServeHTTP(0xc42011e0b0, 0x8b6740, 0xc42013e2a0, 
0xc42000ae00)
/usr/local/go/src/net/http/server.go:2568 +0x92
net/http.(*conn).serve(0xc42008c6e0, 0x8b6d40, 0xc420018680)
/usr/local/go/src/net/http/server.go:1825 +0x612
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2668 +0x2ce
2017/06/01 19:28:54 http: panic serving [::1]:50370: 534 5.7.14 
 Please log in via your web browser and
5.7.14 then try again.
5.7.14  Learn more at
5.7.14  https://support.google.com/mail/answer/78754 w11sm13739759eda.63 - 
gsmtp
goroutine 33 [running]:
net/http.(*conn).serve.func1(0xc4201361e0)
/usr/local/go/src/net/http/server.go:1721 +0xd0
panic(0x6fb7a0, 0xc42043f8e0)
/usr/local/go/src/runtime/panic.go:489 +0x2cf
main.contact(0x8b6740, 0xc4201442a0, 0xc420100b00)
/home/connelblaze/go/src/github.com/connelevalsam/GoWebDev/001/src/sendmail/src/main/main.go:86
 
+0x5d3
net/http.HandlerFunc.ServeHTTP(0x769658, 0x8b6740, 0xc4201442a0, 
0xc420100b00)
/usr/local/go/src/net/http/server.go:1942 +0x44
net/http.(*ServeMux).ServeHTTP(0x8e38e0, 0x8b6740, 0xc4201442a0, 
0xc420100b00)
/usr/local/go/src/net/http/server.go:2238 +0x130
net/http.serverHandler.ServeHTTP(0xc42011e0b0, 0x8b6740, 0xc4201442a0, 
0xc420100b00)
/usr/local/go/src/net/http/server.go:2568 +0x92
net/http.(*conn).serve(0xc4201361e0, 0x8b6d40, 0xc4200f2d40)
/usr/local/go/src/net/http/server.go:1825 +0x612
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2668 +0x2ce


hope to hear from you

On Wednesday, September 2, 2015 at 12:55:26 PM UTC+1, Alexandre Cesaro 
wrote:
>
> Hi,
>
> I just released the second version of Gomail 
> .
>
> There are quite a few backward-incompatible changes 
>  since 
> Gomail v1 but it brings lots of good stuff:
>
>- A great focus was placed on performances. Gomail v2 is way more 
>efficient than Gomail v1 to send a large number of emails (see the 
>Daemon  
>or Newsletter 
> 
>examples).
>- The API is now clearer and way more flexible.
>- The documentation  improved 
>and many examples were added.
>- All existing issues have been closed.
>- The minimum Go version is now 1.2 instead of 1.3. No external 
>dependencies are used with Go 1.5.
>
> More info on Github: https://github.com/go-gomail/gomail
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: [go-nuts] os.Args[0] for program executed by go

[Marvin Renich]

* Santhosh Ram Manohar  [170601 14:03]:
> 
> 
> On Thursday, June 1, 2017 at 10:45:56 AM UTC-7, Jan Mercl wrote:
> >
> > On Thu, Jun 1, 2017 at 7:41 PM Santhosh Ram Manohar  > > wrote:
> >
> > > Args: []string{"self", "selftest"},
> >
> > Here you explicitly pass arsg[0] == "self" and that's what the program 
> > gets.
> >
> 
> Yes, because I already have the program name in "Path". When Run() calls 
> exec syscall i would expect Args[0] to be program name because from the 
> exec point of view its no different from how shell would invoke it, no ?

The documentation for Args says:

// Args holds command line arguments, including the command as Args[0].
// If the Args field is empty or nil, Run uses {Path}.
//
// In typical use, both Path and Args are set by calling Command.

If you use Command, and then print the Args field, you see the proper
setup for Args.  Try https://play.golang.org/p/phIbvlwa8s

Run is using Path as the command to execute and Args as the argv
argument to the underlying execve call (or corresponding Windows API).
The documentation for execve explicitly says that by _convention_ the
first string in argv (argv[0]) should contain the filename associated
with the file being executed.  However, this is just convention, and the
caller of execve is not required to do so.  Not setting argv[0] (i.e.
Args[0]) to the name of the command will normally get undesired results,
since all well-written programs expect argv[0] to be the command name,
not the first command argument.

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] text/template revisit trailing commas for range over map

[Michael Brown]

I see previous threads with a solution to eliminating trailing commas for 
text/template output when ranging over an array. (ie. the {{if 
$index}},{{end}} trick)

I have the exact same problem when ranging over a map and it doesn't appear 
that the array fix will work.

any thoughts?

--
Michael

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] os.Args[0] for program executed by go

[Santhosh Ram Manohar]

On Thursday, June 1, 2017 at 10:45:56 AM UTC-7, Jan Mercl wrote:
>
> On Thu, Jun 1, 2017 at 7:41 PM Santhosh Ram Manohar  > wrote:
>
> > Args: []string{"self", "selftest"},
>
> Here you explicitly pass arsg[0] == "self" and that's what the program 
> gets.
>

Yes, because I already have the program name in "Path". When Run() calls 
exec syscall i would expect Args[0] to be program name because from the 
exec point of view its no different from how shell would invoke it, no ?
 

>
> > What is the reason for this difference only for the binaries executed by 
> the exec package ?
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] os.Args[0] for program executed by go

[Jan Mercl]

On Thu, Jun 1, 2017 at 7:41 PM Santhosh Ram Manohar 
wrote:

> Args: []string{"self", "selftest"},

Here you explicitly pass arsg[0] == "self" and that's what the program gets.

> What is the reason for this difference only for the binaries executed by
the exec package ?

-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: To panic or not to panic, that's the question

[Dave Cheney]

I think that's reasonable, especially as it is not possible to create some sort 
of top level panic handler for init routines.  

Crash only software ✊️

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Latency spike during GC

[Francis]

Xun Liu,

Are you able to post the actual trace here? I appreciate that it contains 
information about your source code, so I can understand if you don't want 
to share it. But I would be very interested to look at the full trace.

Thanks

Francis

On Wednesday, 31 May 2017 21:02:33 UTC+2, Xun Liu wrote:
>
>
>
> On Wed, May 31, 2017 at 11:35 AM, Rick Hudson  > wrote:
>
>> gc 347 @6564.164s 0%: 0.89+518+1.0 ms clock, 28+3839/4091/3959+33 ms cpu, 
>> 23813->23979->12265 MB, 24423 MB goal, 32 P
>>
>> What I'm seeing here is that you have 32 HW threads and you spend 
>> .89+518+1 or 520 ms wall clock in the GC. You also spend  
>> 28+3839+4091+3959+33 
>> or 11950 ms CPU time out of total of 520*32 or 16640 available CPU cycles 
>> while the GC is running. The GC will reserve 25% of the CPU to do its 
>> work. That's 16640*.25 or 4160 ms. If the GC finds any of the remaining 24 
>> threads idle it will aggressively enlist them to do GC work.
>>
>> The graph only shows the 8 HW threads but not the other 24 so it is hard 
>> to tell what is going on with them.
>>
>
> Other cpus are doing user level work. I pasted the whole trace pictures at 
> the end. 
>  
>
>>
>> This may well be related to  issue 12812 
>>  so you might want to read 
>> that thread for more insight.
>>
>>
> I read the issue before which seems to be related. Something special about 
> my trace is many goroutines are GCWaiting, what triggers gc waiting?
> Also is there an easy way to turn off idle gc or assist gc? Not knowing 
> too much details about go gc, I wonder if it's worthy to test how much they 
> play a role in the latency increase -- it feels to me they are too 
> aggressive, e.g. the gc finishes using ~27% of the time it needs to finish: 
> 27% 
> = (23979 - 23813) / (24423 - 23813)
>
> gc 347 @6564.164s 0%: 0.89+518+1.0 ms clock, 28+3839/4091/3959+33 ms cpu, 
> 23813->23979->12265 MB, 24423 MB goal, 32 P
>
>
>
> ​
>
> ​ 
>  
>
>> On Wed, May 31, 2017 at 10:25 AM, Xun Liu > > wrote:
>>
>>> $ go version
>>>
>>> go version go1.8 linux/amd64
>>>
>>> On Wednesday, May 31, 2017 at 7:13:38 AM UTC-7, Ian Lance Taylor wrote:

 [ +rlh, austin ]

 Which version of Go are you running?

 Ian

 On Tue, May 30, 2017 at 10:01 PM, Xun Liu  wrote:

> Hi, we see a clear correlation between GC and latency spike in our Go 
> server. The server uses fairly large amount of memory (20G) and does 
> mostly 
> CPU work. The server runs on a beefy box with 32 cores and the load is 
> pretty light (average CPU 20-30%).  GC kicks in once every 10-20 seconds 
> and whenever it runs we observe pretty big latency spike ranging from 30% 
> to 100% across p50-p90 percentiles (e.g. p90 can jump from 100-120ms to 
> 160-250ms) 
>
> I captured a trace of a gc and noticed the following:
>
> 1. user gorountines seem run longer during gc. This is through ad-hoc 
> check. I don't really know how to get stats to confirm this.
> The gc log is as following (tiny pauses but is very aggressive in 
> assist and idle time)
> gc 347 @6564.164s 0%: 0.89+518+1.0 ms clock, 28+3839/4091/3959+33 ms 
> cpu, 23813->23979->12265 MB, 24423 MB goal, 32 P
>
> 2. during gc, goroutines can queue up. In this particular case there 
> is a stretch of time (~20ms) where we see many goroutines are GCWaiting. 
> See below -- the second row is goroutines with light grey indicating 
> GCWaiting count and light green Runnable.
>
>
>
> 
>
>
> 
>
> Any idea what's going on here? What can I do to reduce the spikes?
>
>  
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Windows - syscall.GetProcessTimes() always shows 0 values.

[Steven Hartland]

All depends on what your going to end up doing.

If code will deal with processes other than itself then pid is arguably 
easier to deal with, however if you only ever deal with your process 
then I would have the NewProcInfo method just setup the handle as you 
had it (reducing the number of syscalls).


QueryProcessCycleTime is not subject to thread time quantum.

You may find the following useful reading:
http://computerperformancebydesign.com/Articles/windows-performance/cpu-utilization/

Regards
Steve

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: idea: generic methods on slices with some generic type support []string.Map(string) []string => [].Map() []

[mhhcbon]

What d be awesome is that the stream like interface 
i have bee thinking about provides a convert method,
so it can answer to this question

given []string how to move on to []int, or,

[]stream.Map(...) -> give me the length of each string

So if the interface contains a magic signature like this,
Conv( conv(), sink) 
sink

It s possible to write,

[]stream.Map(...).Conv(strToLen, []int{}).Sum()

where 
strToLen := func(s string) int { return len(s) }

which, for the fun, could be turned into

strLen := conv(func(v string) int, len)
[]stream.Map(...).Conv(strLen, []int{}).Sum()
// twice conv looks not so good.

On Tuesday, May 30, 2017 at 8:26:35 PM UTC+2, mhh...@gmail.com wrote:
>
> one more case where conv d be awesome,
>
>
> with 
> fnUtils.InputParams() ParamSlice
>
> Where 
> ParamSlice []struct{Name,Type string}
>
> With little help, I can do stuff like
>
> ctxFilter := astutil.FilterParamSlice.ByType(ctxCtx)
> fnUtils.InputParams().Filter(ctxFilter).Map(func(p 
> astutil.Param) astutil.Param {
> ...
> return p
> })
>
> that works, but its not the best factorization possible,
>
> let say the language provide a construct 
> - to make a getter of a type property
> - a deep eq func of a type
>
> such as an approximation would be
>
> typeGet := get.(Param.Type) // func() string
> strEq := conv(func(left func() string) func(right string) 
> bool, eq.(string))
> ctxFilter := strEq(typeGet)
>
> in short
> - make a getter of Param.type => func()string
> - convert string.eq from func(s,s) to a string consumer 
>  => strEq := func(left func() string) func(right string) bool
> - create the value eqer, of a value getter,
> => ctxFilter := strEq(typeGet)
>
> then pass it to a stream/array like data structure (to be defined)
>
> the provider can provides struct only, 
> and the declarer has more ways to handle the declared api.
>
> On Tuesday, May 30, 2017 at 5:14:57 PM UTC+2, mhh...@gmail.com wrote:
>>
>> I just realized what d be awesome is that the language 
>> does not only provide boiler plate []basicType enhanced types,
>> but provides streamed version of them,
>> that d be much better.
>>
>> when you do 
>> [15Million]string.map().filter().count().
>>
>> with a naive implementation over array of those intents,
>> you effectively end up with a copy at each stage,
>> which is not cool.
>>
>> Instead if that d be a stream api, 
>> that d be almost same call, with small changes to be determined, 
>> but a much better allocation,
>> as each item is processed once by each function, one item at a time,
>> you don t need the intermediate (stupid, after all) arrays.
>>
>> [15Million]string.map().filter().count().
>>
>> The difference being 
>> that some func call must become terminal, count() int
>> some must probably become contiguous to > user>, filter() stream
>>
>> can this help to reduce over multiple goroutine?
>>
>> something like ?
>> [15Million]string.chunks(size int, concurrent int).Parrallel(receiver 
>> []string{}.filter() stream).count().
>>
>> count is sequential, it receives results from // filters, 
>> receiver is a worker space (memory+work) aka what a stream ? a []string ?,
>>
>> so ParrallelChunk is (input stream/[]string) stream/[]string
>>
>> why not ?
>>
>> On Tuesday, May 30, 2017 at 4:43:54 PM UTC+2, mhh...@gmail.com wrote:
>>>
>>> I m just gonna add more examples as it come,
>>>
>>> from
>>> hasPrefix := func(prefix string) func(string) bool {
>>> return func(s string) bool { return strings.HasPrefix(s, 
>>> prefix) }
>>> } // this is not cool
>>>
>>> if fnUtils.AllTypes().Filter(hasPrefix("post")).NotEmpty() {
>>> ...
>>> }
>>>
>>> to
>>> hasPrefix, _ := conv(func(prefix string) func(s string) 
>>> bool, strings.HasPrefix)
>>> // does this make sense given 
>>> https://golang.org/pkg/strings/#HasPrefix ? 
>>>
>>> if fnUtils.AllTypes().Filter(hasPrefix("post")).NotEmpty() {
>>> ...
>>> }
>>>
>>> On Friday, May 26, 2017 at 2:47:45 PM UTC+2, mhh...@gmail.com wrote:

 oops... mistake in it.


 printS, err := conv(func(s string, err error), fmt.Println) or 
 panic(err)
 _, err := []string{"hello}.Map(strings.
 ToUpper).MustEach(printS) or panic(err)

 count, err := conv(func(s string) n int, fmt.Println) or panic(err)
 n := []string{"hello}.Map(strings.ToUpper).Sum(count)

 count, err := conv(func(s string) (n int, err error), fmt.Println) or 
 panic(err)
 n, err := []string{"hello}.Map(strings.ToUpper).MustSum(count) or 
 panic(err)

 more like this, take advantage of return type and names to do more 
 conversion.

 On Friday, May 26, 2017 at 2:45:33 PM UTC+2, mhh...@gmail.com wrote:
>
> or this,
>
> printS, err := conv(func(s string, err 

[go-nuts] Re: some problems with ECDSA ScalarMult

[张伟]

The full codes:

package main

import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"encoding/hex"
"fmt"
"math/big"
)

func main() {
key2, _ := 
hex.DecodeString("646d22e7aee42d44bd15cdf58006359283e1da83c2670b25d44906d03e9ed4eb")
X2, _ := 
new(big.Int).SetString("3842ab25368451a004bb57f2008106aa18578a2887f5a34d17c92a0f6c22241f",
 
16)
Y2, _ := 
new(big.Int).SetString("72be1183f839f5a9e8841873cd074151da8dd5881fffc7668e2c189c142dad4c",
 
16)

priv2, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
aeskey2, _ := priv2.Curve.ScalarMult(X2, Y2, key2)
fmt.Printf("to: %x\n", aeskey2)

}




在 2017年6月1日星期四 UTC+8下午12:01:49，张伟写道：
>
> I am using ECC ScalarMult to get a point. When I use the amd64 accelerated 
> version, the result is wrong; by using the generic version, the result is 
> normal.
> My code is here:
>
> k, _ := 
> hex.DecodeString("646d22e7aee42d44bd15cdf58006359283e1da83c2670b25d44906d03e9ed4eb")
> X, _ := 
> new(big.Int).SetString("3842ab25368451a004bb57f2008106aa18578a2887f5a34d17c92a0f6c22241f",
>  
> 16)
> Y, _ := 
> new(big.Int).SetString("72be1183f839f5a9e8841873cd074151da8dd5881fffc7668e2c189c142dad4c",
>  
> 16)
>
> priv, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
> p, _ := priv.Curve.ScalarMult(X, Y, k)
> fmt.Printf("%x\n", p)
>
>
> I need 
> "D68B74C852C7F4D271AB7ADC8B451943688DCDFA3F4CA37841CE325461AEEB1B169D519667ABB11651FEDDBCF1F4C52A4FF8A93ACE1C270E8523FDC842AFF005",
> but I get 
> "240AD1096C1C7CBF180067AC3F214DE462881FF3FCFC77B7516762E2BD2AA931D7C99D8200DD54B4182D81C77E720151750FAF26FAB6CF89603C57B46B2BDD85"
>
>
> Thanks for your help!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: To panic or not to panic, that's the question

[mhhcbon]

understood the same, was going to give same example. 
panic at init is ok to me while it look likes 
a compile error
a system requirement error



On Wednesday, May 31, 2017 at 9:41:37 PM UTC+2, Pierre Durand wrote:
>
> I use these functions in init():
> https://golang.org/pkg/html/template/#Must
> https://golang.org/pkg/regexp/#MustCompile
>
> Le mercredi 31 mai 2017 21:24:58 UTC+2, Peter Kleiweg a écrit :
>>
>> If a package can't be used because some precondition can't be fulfilled, 
>> you can use a panic. Then the whole program will crash at start-up.
>>
>> The alternative would be to not panic in the init, but have all function 
>> calls return an error. If the package is used in a large program, part of 
>> it may still be functional.
>>
>> Using panic in an init function of a package, do or don't?
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: did anyone tried // template exec ?

[mhhcbon]

I see, interesting, thanks for that.

Note, those are different problems.

One is about trying to cheaply speed up template execution by //,
I still wonder if that can be worthy,
i foresee with the future translation package it might because of the 
additional call costs,
but you know side effects, allocations, contentions... stuff like that.. 
and you are back to (unfinished...) alternatives ;)

The second is about handling data injection into the template.

Where i do have several critics too,
I d simply won t put my services in a kv[string]interface{}, 
better to use another language if that is the final solution.

To compare with context, at least context is a kv[interface{}]interface{},
which you can take advantage to be protective.

On large scale, this approach is difficult to manage.



Anyways some extended thoughts.

For that specific case it happens i m working on what might be a solution,
like the article you presented it does code introspection 
so the computer does the *** job for us.

I use typed struct to do data injection, 

of services
type TemplateData struct {
SvcWhatever ServiceTypeToLookupFor
}

of other view components services
type TemplateData struct {
ViewWhatever ViewTypeToLookupFor
}

so far it seems to be perfect fit to rely on the type system rather than 
raw text.

Then i do some preprocessing (rather than reflect) to bind that 
automatically.

With more help from the computer i also decoupled 
the transport controller and the view controllerS,
so i ll never have this stupid global head context
to manage every details of the page to be generated.

views won t be that stupid dead code anymore, 
it will manage its input to produce the desired output.
So everyone is more autonomous on his task assignment 
and the code is generally more localized which looks better to me.

Last, i d really like the same article techniques applied to translation 
management.
The translation are always sparse across the application,
every time its a pain to handle, either way, 
by declare ahead and use in the template, 
or use in template than declare in the translations.

I d rather have a consume first approach with the help of 
a tool that inspects the templates for translation call, 
and process them to dedicated dictionaries to answer
several useful questions,
howto
Insert translation of ID I for language N?
Where is that translation ID used in the application?
Does this translation requires plural support in the app ?
Prepare the file to update/create the translations?
etc etc

On Thursday, June 1, 2017 at 8:05:16 AM UTC+2, Michael Brown wrote:
>
> I posted on another thread today one solution for this (Subj: "How to 
> parallelize text/template long-running functions")
>
> I got something simple working using a two-pass template execution, and 
> somebody else pointed me to 
> https://blog.gopheracademy.com/advent-2014/soy-programmable-templates/ 
> (which I haven't yet examined).
> --
> Michael
>
> On Tuesday, May 30, 2017 at 3:09:35 AM UTC-5, mhh...@gmail.com wrote:
>>
>> hi,
>>
>> wonder, if anyone,
>> - tried to make an implementation of template that works in // ?
>> - was it successful ?
>> - useful for perf ?
>>
>> thanks
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: did anyone tried // template exec ?

[Michael Brown]

I posted on another thread today one solution for this (Subj: "How to 
parallelize text/template long-running functions")

I got something simple working using a two-pass template execution, and 
somebody else pointed me 
to https://blog.gopheracademy.com/advent-2014/soy-programmable-templates/ 
(which I haven't yet examined).
--
Michael

On Tuesday, May 30, 2017 at 3:09:35 AM UTC-5, mhh...@gmail.com wrote:
>
> hi,
>
> wonder, if anyone,
> - tried to make an implementation of template that works in // ?
> - was it successful ?
> - useful for perf ?
>
> thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to parallelize text/template long running functions?

[Michael Brown]

Great! If I had checked this thread when you posted this I (probably) could 
have saved myself 3 hours of work.

I got it working with a two-pass scheme using text/template and some 
channels.

Here is what I came up with:

package main


import (

"text/template"

"bytes"

"context"

"fmt"

"github.com/google/uuid"

"time"

)


type resultpair struct {

key   string

value string

}



func getFuncMap(ctx context.Context, resultsQueue chan resultpair) (*int, 
template.FuncMap) {

totalJobs := 0

var funcMap = template.FuncMap{

"sleep": func() string {

totalJobs = totalJobs + 1

token := uuid.New()

go func() {

time.Sleep(1 * time.Second)

resultsQueue <- resultpair{token.String(), "REAL_VALUE!"}

}()

return "{{ getOutput \"" + token.String() + "\" }}"

},

}


return , funcMap

}


func getFuncMapNested(ctx context.Context, output map[string]string) 
template.FuncMap {

var funcMapNested = template.FuncMap{

"getOutput": func(input string) string { return output[input] },

}

return funcMapNested

}


func main() {

initial := "{{sleep}} {{sleep}} {{sleep}}"


resultsQueue := make(chan resultpair)

outputQueue := make(chan map[string]string)

// totalJobs is decieving: only ever accessed by one thread at a time, 
so shouldn't need locking (I think)

totalJobs, funcMap := getFuncMap(context.TODO(), resultsQueue)


fmt.Printf("About to execute first template: %s\n", initial)

fmt.Printf("TOTAL JOBS: %d\n", *totalJobs)

tmpl, _ := template.New("test").Funcs(funcMap).Parse(initial)

var buf bytes.Buffer

tmpl.Execute(, nil)

fmt.Printf("Got translated template: %s\n", buf.String())

fmt.Printf("TOTAL JOBS: %d\n", *totalJobs)


go func(totalJobs *int) {

var results map[string]string

results = make(map[string]string)


for i := 0; i < *totalJobs; i++ {

res := <-resultsQueue

results[res.key] = res.value

}

outputQueue <- results

close(outputQueue)

}(totalJobs)


output := <-outputQueue

close(resultsQueue)

fmt.Printf("Output of the goroutine: %s\n", output)


funcMapNested := getFuncMapNested(context.TODO(), output)

tmpl2, _ := 
template.New("nested").Funcs(funcMapNested).Parse(buf.String())

var buf2 bytes.Buffer

tmpl2.Execute(, nil)


fmt.Printf("results: %s\n", buf2.String())

}


OUTPUT:


$ time go run ./commands/try.go

About to execute first template: {{sleep}} {{sleep}} {{sleep}}

TOTAL JOBS: 0

Got translated template: {{ getOutput 
"bc7dcfa0-89d9-45e9-bd40-eb2db6f51db0" }} {{ getOutput 
"f2539f15-378b-408d-8c6e-d3822e985a6b" }} {{ getOutput 
"56c9d239-d08d-43e8-80de-dd97ef157b6a" }}

TOTAL JOBS: 3

Output of the goroutine: 
map[bc7dcfa0-89d9-45e9-bd40-eb2db6f51db0:REAL_VALUE! 
f2539f15-378b-408d-8c6e-d3822e985a6b:REAL_VALUE! 
56c9d239-d08d-43e8-80de-dd97ef157b6a:REAL_VALUE!]

results: REAL_VALUE! REAL_VALUE! REAL_VALUE!


real0m1.319s

user0m0.278s

sys 0m0.098s

On Wednesday, May 31, 2017 at 9:09:51 PM UTC-5, robfig wrote:
>
> We do this exact thing except using closure templates
> https://blog.gopheracademy.com/advent-2014/soy-programmable-templates/
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.