[go-nuts] Re: High memory usage of math/big.putNat for web application

[Raffaele Di Fazio]

I haven't tried tip yet, but I saw this patch yesterday. It looks like this 
might help as I'm not using these pools directly. Over time though, I would 
expect the GC to free the memory allocated. It's true that I see much lower 
values in the profiler for the in_use memory, but I wonder why my machine 
keeps saying that the application is using quite some memory (400 MB for a 
simple web app seems quite a lot)... maybe the GC collects the memory but 
does not return it to the system? 

@Andrey: I might create a gist, unfortunately I can't share the full 
codebase. The initialization though, looks mostly like 
that: https://github.com/zalando/chimp/blob/master/api/server.go#L124 . I 
wonder if setting proper read and write timeouts and disabling keepalives 
might help. 



On Saturday, October 15, 2016 at 8:43:44 PM UTC+2, alb.do...@gmail.com 
wrote:
>
> Does this happens on tip too? There was a recent CL that
> modified the code of the nat pool; see
>
> https://go-review.googlesource.com/#/c/30613/
>
> exp. the "Eliminate allocation in divLarge nat pool" part.
>
>
> Il giorno sabato 15 ottobre 2016 16:28:01 UTC+2, Raffaele Di Fazio ha 
> scritto:
>>
>> Hi, 
>> I have a web application that over time uses more and more memory. This 
>> is the output of pprof of the heap: 
>>
>> go tool pprof -alloc_space lushan-server https:
>> //localhost:8083/debug/pprof/heap
>> Fetching profile from https://localhost:8083/debug/pprof/heap
>> Saved profile in /Users/rdifazio/pprof/pprof.lushan-server.localhost:
>> 8083.alloc_objects.alloc_space.022.pb.gz
>> Entering interactive mode (type "help" for commands)
>> (pprof) top
>> 43.06MB of 67.07MB total (64.20%)
>> Dropped 4 nodes (cum <= 0.34MB)
>> Showing top 10 nodes out of 188 (cum >= 1.50MB)
>>   flat  flat%   sum%cum   cum%
>>   26MB 38.76% 38.76%   26MB 38.76%  math/big.putNat
>>3MB  4.48% 43.24% 3.50MB  5.22%  encoding/json.(*decodeState).
>> objectInterface
>> 2.50MB  3.73% 46.97% 2.50MB  3.73%  crypto/tls.(*Conn).write
>>2MB  2.98% 49.96%2MB  2.98%  crypto/tls.(*block).reserve
>>2MB  2.98% 52.94%10.50MB 15.66%  encoding/json.Unmarshal
>> 1.55MB  2.31% 55.25% 1.55MB  2.31%  regexp.(*bitState).reset
>> 1.50MB  2.24% 57.49% 9.50MB 14.17%  github.com/go-openapi/spec.(*
>> Schema).UnmarshalJSON
>> 1.50MB  2.24% 59.72%27.50MB 41.00%  math/big.nat.divLarge
>> 1.50MB  2.24% 61.96% 5.50MB  8.20%  math/big.nat.expNN
>> 1.50MB  2.24% 64.20% 1.50MB  2.24%  crypto/sha512.New384
>> (pprof)
>>
>>
>> ROUTINE  math/big.putNat in /usr/local/Cellar/go/
>> 1.7.1/libexec/src/math/big/nat.go
>>   26MB   26MB (flat, cum) 38.76% of Total
>>  .  .550: }
>>  .  .551: return z.make(n)
>>  .  .552:}
>>  .  .553:
>>  .  .554:func putNat(x nat) {
>>   26MB   26MB555: natPool.Put(x)
>>  .  .556:}
>>  .  .557:
>>  .  .558:var natPool sync.Pool
>>  .  .559:
>>  .  .560:// q = (uIn-r)/v, with 0 <= r < y
>>
>>
>>
>> The memory allocated in math/big.putNat seems to increase over time, 
>> generating a very high usage of memory for a web applications that is 
>> executing very few requests per second. I wonder why and how I can better 
>> analyze this issue. Please notice that this happens only when serving 
>> HTTPS. 
>>
>> I'm currently using go 1.7 and the app itself uses the gin web framework. 
>>
>> Thanks in advance! 
>>
>> Raffaele 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Native activity on android / ios

[Ged Wed]

I am using gomobile. It's very compelling.

I want to use the NativeActivity to write a daemon / service in 100℅ golang.
I need to expose a http2 service with it.

The problem :  opengl style gomobile apps are possible, but can't work out how 
to use NativeAxtivity to expose a http service.

Anyone got any advice ?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: High memory usage of math/big.putNat for web application

[Raffaele Di Fazio]

Here some pprof result using go 1.7.1 first and tip later: 


 go tool pprof -alloc_space lushan-server https:
//localhost:8083/debug/pprof/heap
Fetching profile from https://localhost:8083/debug/pprof/heap
Saved profile in /Users/rdifazio/pprof/pprof.lushan-server.localhost:
8083.alloc_objects.alloc_space.078.pb.gz
Entering interactive mode (type "help" for commands)
(pprof) top
1294.76MB of 1549.02MB total (83.59%)
Dropped 117 nodes (cum <= 7.75MB)
Showing top 10 nodes out of 89 (cum >= 20.50MB)
  flat  flat%   sum%cum   cum%
  865.53MB 55.88% 55.88%   866.03MB 55.91%  math/big.putNat
   99.04MB  6.39% 62.27%   965.57MB 62.33%  math/big.nat.divLarge
   89.05MB  5.75% 68.02%91.05MB  5.88%  math/big.nat.mul
   51.51MB  3.33% 71.34%51.51MB  3.33%  math/big.nat.montgomery
   51.01MB  3.29% 74.64%   928.54MB 59.94%  math/big.(*Int).GCD
   38.04MB  2.46% 77.09%38.04MB  2.46%  crypto/tls.(*block).reserve
   35.05MB  2.26% 79.36%35.05MB  2.26%  crypto/tls.(*Conn).write
   23.01MB  1.49% 80.84%   218.09MB 14.08%  math/big.nat.expNN
   22.03MB  1.42% 82.26%22.03MB  1.42%  crypto/elliptic.(*p256Point).
p256ScalarMult
   20.50MB  1.32% 83.59%20.50MB  1.32%  crypto/sha256.New
(pprof) %   


  lushan-server master % htop
lushan-server master % htop
lushan-server master % go tool pprof -alloc_space lushan-server https:
//localhost:8083/debug/pprof/heap
Fetching profile from https://localhost:8083/debug/pprof/heap
http fetch https://localhost:8083/debug/pprof/heap: Get 
https://localhost:8083/debug/pprof/heap: dial tcp [::1]:8083: getsockopt: 
connection refused
lushan-server master % go tool pprof -alloc_space lushan-server https:
//localhost:8083/debug/pprof/heap
Fetching profile from https://localhost:8083/debug/pprof/heap
Saved profile in /Users/rdifazio/pprof/pprof.lushan-server.localhost:
8083.alloc_objects.alloc_space.079.pb.gz
Entering interactive mode (type "help" for commands)
(pprof) top
366.69MB of 536.79MB total (68.31%)
Dropped 80 nodes (cum <= 2.68MB)
Showing top 10 nodes out of 106 (cum >= 170.57MB)
  flat  flat%   sum%cum   cum%
   76.54MB 14.26% 14.26%82.04MB 15.28%  math/big.nat.mul
   73.53MB 13.70% 27.96%74.03MB 13.79%  math/big.nat.divLarge
   50.51MB  9.41% 37.37%50.51MB  9.41%  math/big.nat.montgomery
   32.51MB  6.06% 43.42%70.01MB 13.04%  math/big.(*Int).GCD
   30.53MB  5.69% 49.11%30.53MB  5.69%  crypto/tls.(*block).reserve
   27.01MB  5.03% 54.14%27.01MB  5.03%  math/big.nat.add
   26.53MB  4.94% 59.08%26.53MB  4.94%  crypto/tls.(*Conn).write
   21.53MB  4.01% 63.09%21.53MB  4.01%  crypto/elliptic.(*p256Point).
p256ScalarMult
   14.50MB  2.70% 65.80%14.50MB  2.70%  crypto/sha256.New
   13.50MB  2.52% 68.31%   170.57MB 31.78%  math/big.nat.expNN
(pprof)




The improvement is definitely noticeable. The questions from before are 
still open though :-) 



On Sunday, October 16, 2016 at 11:39:02 AM UTC+2, Raffaele Di Fazio wrote:
>
> I haven't tried tip yet, but I saw this patch yesterday. It looks like 
> this might help as I'm not using these pools directly. Over time though, I 
> would expect the GC to free the memory allocated. It's true that I see much 
> lower values in the profiler for the in_use memory, but I wonder why my 
> machine keeps saying that the application is using quite some memory (400 
> MB for a simple web app seems quite a lot)... maybe the GC collects the 
> memory but does not return it to the system? 
>
> @Andrey: I might create a gist, unfortunately I can't share the full 
> codebase. The initialization though, looks mostly like that: 
> https://github.com/zalando/chimp/blob/master/api/server.go#L124 . I 
> wonder if setting proper read and write timeouts and disabling keepalives 
> might help. 
>
>
>
> On Saturday, October 15, 2016 at 8:43:44 PM UTC+2, alb.do...@gmail.com 
> wrote:
>>
>> Does this happens on tip too? There was a recent CL that
>> modified the code of the nat pool; see
>>
>> https://go-review.googlesource.com/#/c/30613/
>>
>> exp. the "Eliminate allocation in divLarge nat pool" part.
>>
>>
>> Il giorno sabato 15 ottobre 2016 16:28:01 UTC+2, Raffaele Di Fazio ha 
>> scritto:
>>>
>>> Hi, 
>>> I have a web application that over time uses more and more memory. This 
>>> is the output of pprof of the heap: 
>>>
>>> go tool pprof -alloc_space lushan-server https:
>>> //localhost:8083/debug/pprof/heap
>>> Fetching profile from https://localhost:8083/debug/pprof/heap
>>> Saved profile in /Users/rdifazio/pprof/pprof.lushan-server.localhost:
>>> 8083.alloc_objects.alloc_space.022.pb.gz
>>> Entering interactive mode (type "help" for commands)
>>> (pprof) top
>>> 43.06MB of 67.07MB total (64.20%)
>>> Dropped 4 nodes (cum <= 0.34MB)
>>> Showing top 10 nodes out 

[go-nuts] Proposal: add "future" internal type (similar to channel)

[Sokolov Yura]

"future" is commonly used synchronization abstraction.

It could be implemented in a library, using mutex, channel and interface.
Example: 
https://github.com/Workiva/go-datastructures/blob/master/futures/selectable.go

But obviously, semantically future is just a channel with buffer of 
capacity 1, but receivers do not pop
value from a buffer, but instead every receiver receive same value. And 
"filling" future awakes all
receivers simultaneously, similar to "closing of channel".

So I propose to introduce "future" as a same internal type as a "channel".
It will share representation and lot of code with buffered channel (of 
capacity 1).

https://github.com/golang/go/issues/17466

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Duplicate File Checker Performance

[Michael Jones]

Sri G,

 

How does this time compare to my “Dup” program? I can’t test for you…since it 
is your filesystem…but I thought I had it going about as fast as possible a few 
years ago when I wrote that one.

 

https://github.com/MichaelTJones/dup

 

Michael

 

From:  on behalf of Sri G 

Date: Saturday, October 15, 2016 at 6:46 PM
To: golang-nuts 
Subject: [go-nuts] Re: Duplicate File Checker Performance

 

Thanks. Made the go code similar to python using CopyBuffer with a block size 
of 65536. 

 

buf := make([]byte, 65536)



if _, err := io.CopyBuffer(hash, file, buf); err != nil {

fmt.Println(err)

}

 

Didn't make too much of a difference, was slightly faster.

 

What got it to the same place was running ComputeHash in the same goroutine as 
the Walk function vs its own go routine for each file

 

+ComputeHash(path, info, queue, wg)

-go ComputeHash(path, info, queue, wg)

 

 

2.88s user 0.98s system 23% cpu 16.086 total, memory usage ~ 7MB

Here's the before and after pprof webs:

 

BEFORE with 'go ComputeHash(...):

 

 

 

AFTER with 'ComputeHash(...):

 

 

 

Since disk read are SOO much slower, computing the hash for each file in its 
own goroutine caused a huge slowdown.. 

 

btw this is on a RAID10, with SSD: 

 

Old code SSD: 3.31s user 17.87s system 244% cpu 8.667 total

 

New code SDD: 2.88s user 0.84s system 69% cpu 5.369 total

 

Shows you can throw hardware at a problem BUT the old code locks up my system 
momentarily..

 


On Saturday, October 15, 2016 at 3:27:38 PM UTC-4, Kevin Malachowski wrote:

Sorry, I meant that calling Write on the hash type might be slower if it's 
called more often.

(I'm on mobile right now. When I get back to a keyboard I'll try to come up 
with an example)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] cgo: support for exporting structs?

[Andy]

Richard:

If you're asking about:

  "cmd/cgo: annotate named return struct members in comments "
  https://go-review.googlesource.com/#/c/13061/

It looks like that was merged at some point, but I'm not sure.

I abandoned my experiments with go/cgo when some of the limitations became
clear.  I wouldn't ultimately be able to do what I'd wanted to with it.

---
Andy Maloney  //  https://asmaloney.com
twitter ~ @asmaloney 


On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes 
wrote:

> Andy,
>
> What's the status of getting this change into Go? I have an issue that
> could benefit from this particular support and was curious if/when I'd be
> able to use it.
>
> Thanks!
>
> - Rich
>
>
> On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:
>>
>> I imagine it will take some back-and-forth with you to get it into shape
>> - make sure it's idiomatic and fits with the current structure - so I'll
>> wait until after 1.5 is out.
>>
>> It requires changes I put in another CL (https://go-review.googlesourc
>> e.com/#/c/13061/ ).  How do I handle that when I want to create a new
>> CL?  Just branch off that one?  Or do just wait until the other one is
>> accepted and merged?
>>
>> On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:
>>>
>>> On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney  wrote:
>>> > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor wrote:
>>> >>
>>> >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney 
>>> wrote:
>>> >> >
>>> >> > after all the Decls are added to the Package in Record() (in
>>> main.go),
>>> >> > call
>>> >> > a function which looks at each of the exported functions and uses
>>> the
>>> >> > Decls
>>> >> > to create a map to store some information about struct parameters &
>>> >> > results
>>> >> > in cgoType(), use the map that was just created to lookup the name
>>> of
>>> >> > the
>>> >> > struct and its size (which we calculated and stored in the map)
>>> and, if
>>> >> > we
>>> >> > find it, return it
>>> >> > at the end of writeExportHeader(), write out typedefs for each of
>>> the
>>> >> > structs in the map
>>> >>
>>> >> I'm not quite sure what you mean here.  It seems to me that you can
>>> >> treat any struct used by an exported function as though the Go code
>>> >> said "C.struct_foo".  That should define in Go like other cgo
>>> >> structs.
>>> >
>>> > I'm not seeing how to do this, but does that mean from my example
>>> above I'd
>>> > end up with this in the header:
>>> >
>>> > typedef struct {
>>> > GoIntr0;
>>> > GoStringr1;
>>> > } struct_Bar;
>>> >
>>> >
>>> > extern struct_Bar Foo();
>>> >
>>> > (And I'd lose the names of the members like I currently do in the
>>> return
>>> > structs?)
>>>
>>> Sorry, I was confused.  You have structs defined in Go, and you need
>>> them to be defined in C.  For some reason I was thinking about this
>>> the other way around.
>>>
>>> Sure, something like what you described sounds reasonable.
>>>
>>>
>>> >> > Right now I just look at structs, but it would make sense to me
>>> that if
>>> >> > you
>>> >> > declare
>>> >> > type Foo int
>>> >> > and you use it as a result to a function we should see
>>> >> > typedef GoInt Foo;
>>> >> > in the header file.  Would that make sense to add too?
>>> >>
>>> >> I doubt it.  C typedefs are not Go typedefs.  Go's types are much
>>> >> stricter.  I expect that changing this would break currently working
>>> >> code.
>>> >
>>> >
>>> > Wouldn't this just be preserving the name of the type in the C code?
>>> > Whether you call it a GoInt or a Foo in this case should be the same -
>>> it's
>>> > just an alias in C.  Not sure I see how that would break anything.
>>>
>>> Same confusion on my part.  I'm not sure the typedef names help much
>>> but I suppose it is fine.
>>>
>>> Ian
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: High memory usage of math/big.putNat for web application

[andrey mirtchovski]

> Here some pprof result using go 1.7.1 first and tip later:

If I understand you correctly, you still have questions about memory
management, but the nature of the question has changed from "why does
this function keep increasing memory consumption" to "why does my
program consume so much memory"? I think the former was answered. The
latter boils down to the nature of Go memory management.

A variable called GOGC controls when the next collection will begin.
"A collection is triggered when the ratio of freshly allocated data to
live data remaining after the previous collection reaches this
percentage." GOGC by default is 100, so you roughly double your
used/allocated memory before you start freeing up objects. To monitor
the collection cycles you can run your code with
"GODEBUG=gctrace=1"[1] (see link at bottom for documentation).

Go normally does not return all deallocated memory to the host
immediately after deallocation. That would not be very efficient.
Instead, scavenging happens on certain intervals. gctrace=1 will
output information about that too, the documentation of the format is
again at the link at the bottom.

--
1: https://golang.org/pkg/runtime/

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: High memory usage of math/big.putNat for web application

[Raffaele Di Fazio]

Thank you for your answer, I now understand a bit better what is going on.

On Sun, Oct 16, 2016 at 7:56 PM, andrey mirtchovski 
wrote:

> > Here some pprof result using go 1.7.1 first and tip later:
>
> If I understand you correctly, you still have questions about memory
> management, but the nature of the question has changed from "why does
> this function keep increasing memory consumption" to "why does my
> program consume so much memory"? I think the former was answered. The
> latter boils down to the nature of Go memory management.
>
> A variable called GOGC controls when the next collection will begin.
> "A collection is triggered when the ratio of freshly allocated data to
> live data remaining after the previous collection reaches this
> percentage." GOGC by default is 100, so you roughly double your
> used/allocated memory before you start freeing up objects. To monitor
> the collection cycles you can run your code with
> "GODEBUG=gctrace=1"[1] (see link at bottom for documentation).
>
> Go normally does not return all deallocated memory to the host
> immediately after deallocation. That would not be very efficient.
> Instead, scavenging happens on certain intervals. gctrace=1 will
> output information about that too, the documentation of the format is
> again at the link at the bottom.
>
> --
> 1: https://golang.org/pkg/runtime/
>



-- 
Raffaele Di Fazio

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Duplicate File Checker Performance

[Sri G]

This isn't exactly the same because I deleted some files but it shouldn't 
really matter.  

Switched to md5..

--- a/dup.go
+++ b/dup.go
@@ -3,7 +3,7 @@ package main
-   "crypto/sha256"
+   "crypto/md5"

@@ -207,8 +207,8 @@ func main() {
+type Hash [16]byte // appropriate for MD5
+// type Hash [32]byte // appropriate for SHA-256

 func hashFile(p string, hash []byte, prefix int64) (count int64) {
@@ -221,8 +221,8 @@ func hashFile(p string, hash []byte, prefix int64) 
(count int64) {
+   hasher := md5.New() // select MD5 in concert with "Hash" above
+   // hasher := sha256.New() // select SHA-256 in concert with "Hash" 
above


Checking only same sized files is huge speed up (82x less bytes checked)

016/10/16  14:33:51  total:  566 files ( 100.00%),1667774744 
bytes ( 100.00%)
2016/10/16 14:33:51   examined:9 files (   1.59%),  20271440 
bytes (   1.22%) in 0.4209 seconds
2016/10/16 14:33:51 duplicates:9 files (   1.59%),  20271440 
bytes (   1.22%)

Checking the first 4KB of files and only hashing if they are the same is 
another cool optimization (check avg. 768x less bytes in my case). Really 
nice Michael.

With workers = 8:

RAID10:* 0.05s user 0.04s system 37% cpu 0.231 total, couldn't check memory 
usage but its probably negligible*
SSD:*0.05s user 0.04s system 59% cpu 0.137 total*

Since SSD's and my filesystem are optimized for 4K random reads, it makes 
sense to use multiple threads/goroutines.

Optimal # of workers=9 on RAID 10: *0.05s user 0.04s system 40% cpu 0.220 
total*
on SSD workers = 8~9:  *0.04s user 0.04s system 68% cpu 0.117 
total*

Not so much when you're doing a full sequential read. Because I use the md5 
for other purposes, the entire file must be hashed, so sadly I cant use 
these optimizations.

On Sunday, October 16, 2016 at 1:26:24 PM UTC-4, Michael Jones wrote:
>
> Sri G,
>
>  
>
> How does this time compare to my “Dup” program? I can’t test for you…since 
> it is your filesystem…but I thought I had it going about as fast as 
> possible a few years ago when I wrote that one.
>
>  
>
> https://github.com/MichaelTJones/dup
>
>  
>
> Michael
>
>  
>
> *From: *> on behalf of Sri G <
> sriakhil...@gmail.com >
> *Date: *Saturday, October 15, 2016 at 6:46 PM
> *To: *golang-nuts >
> *Subject: *[go-nuts] Re: Duplicate File Checker Performance
>
>  
>
> Thanks. Made the go code similar to python using CopyBuffer with a block 
> size of 65536. 
>
>  
>
> buf := make([]byte, 65536)
>
> 
>
> if _, err := io.CopyBuffer(hash, file, buf); err != nil {
>
> fmt.Println(err)
>
> }
>
>  
>
> Didn't make too much of a difference, was slightly faster.
>
>  
>
> What got it to the same place was running ComputeHash in the same 
> goroutine as the Walk function vs its own go routine for each file
>
>  
>
> +ComputeHash(path, info, queue, wg)
>
> -go ComputeHash(path, info, queue, wg)
>
>  
>
>  
>
> *2.88s user 0.98s system 23% cpu 16.086 total, memory usage ~ 7MB*
>
> Here's the before and after pprof webs:
>
>  
>
> BEFORE with 'go ComputeHash(...):
>
>  
>
>
> 
>
>  
>
>  
>
> AFTER with 'ComputeHash(...):
>
>  
>
>
> 
>
>  
>
>  
>
> Since disk read are SOO much slower, computing the hash for each file in 
> its own goroutine caused a huge slowdown.. 
>
>  
>
> btw this is on a RAID10, with SSD: 
>
>  
>
> Old code SSD:* 3.31s user 17.87s system 244% cpu 8.667 total*
>
>  
>
> New code SDD:* 2.88s user 0.84s system 69% cpu 5.369 total*
>
>  
>
> Shows you can throw hardware at a problem BUT the old code locks up my 
> system momentarily..
>
>  
>
>
> On Saturday, October 15, 2016 at 3:27:38 PM UTC-4, Kevin Malachowski wrote:
>
> Sorry, I meant that calling Write on the hash type might be slower if it's 
> called more often.
>
> (I'm on mobile right now. When I get back to a keyboard I'll try to come 
> up with an example)
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] cgo: support for exporting structs?

[Andy]

Ah - ok.

As I recall, I submitted a CR for it but Ian pointed out an edge case I
needed to handle. I never got to it because I changed direction and it
didn't seem like anyone else was interested in this modification at the
time.

I can dig out the diffs from my previous attempt if that would help get you
started - let me know.

---
Andy Maloney  //  https://asmaloney.com
twitter ~ @asmaloney 


On Sun, Oct 16, 2016 at 3:07 PM, Richard Wilkes  wrote:

> Hi, Andy.
>
> Thanks for replying. No, I was interested in the change to allow a struct
> defined in Go to be passed to C, even if it only supported basic types.
> Sounds like that part never made it in — certainly Go 1.7.1 doesn’t seem to
> allow me to do such things, which is unfortunate.
>
> - Rich
>
>
>
> On Oct 16, 2016, at 10:23 AM, Andy  wrote:
>
> Richard:
>
> If you're asking about:
>
>   "cmd/cgo: annotate named return struct members in comments "
>   https://go-review.googlesource.com/#/c/13061/
>
> It looks like that was merged at some point, but I'm not sure.
>
> I abandoned my experiments with go/cgo when some of the limitations became
> clear.  I wouldn't ultimately be able to do what I'd wanted to with it.
>
> ---
> Andy Maloney  //  https://asmaloney.com
> twitter ~ @asmaloney 
>
>
> On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes 
> wrote:
>
>> Andy,
>>
>> What's the status of getting this change into Go? I have an issue that
>> could benefit from this particular support and was curious if/when I'd be
>> able to use it.
>>
>> Thanks!
>>
>> - Rich
>>
>>
>> On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:
>>>
>>> I imagine it will take some back-and-forth with you to get it into shape
>>> - make sure it's idiomatic and fits with the current structure - so I'll
>>> wait until after 1.5 is out.
>>>
>>> It requires changes I put in another CL (https://go-review.googlesourc
>>> e.com/#/c/13061/ ).  How do I handle that when I want to create a new
>>> CL?  Just branch off that one?  Or do just wait until the other one is
>>> accepted and merged?
>>>
>>> On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:

 On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney 
 wrote:
 > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor
 wrote:
 >>
 >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney 
 wrote:
 >> >
 >> > after all the Decls are added to the Package in Record() (in
 main.go),
 >> > call
 >> > a function which looks at each of the exported functions and uses
 the
 >> > Decls
 >> > to create a map to store some information about struct parameters
 &
 >> > results
 >> > in cgoType(), use the map that was just created to lookup the name
 of
 >> > the
 >> > struct and its size (which we calculated and stored in the map)
 and, if
 >> > we
 >> > find it, return it
 >> > at the end of writeExportHeader(), write out typedefs for each of
 the
 >> > structs in the map
 >>
 >> I'm not quite sure what you mean here.  It seems to me that you can
 >> treat any struct used by an exported function as though the Go code
 >> said "C.struct_foo".  That should define in Go like other cgo
 >> structs.
 >
 > I'm not seeing how to do this, but does that mean from my example
 above I'd
 > end up with this in the header:
 >
 > typedef struct {
 > GoIntr0;
 > GoStringr1;
 > } struct_Bar;
 >
 >
 > extern struct_Bar Foo();
 >
 > (And I'd lose the names of the members like I currently do in the
 return
 > structs?)

 Sorry, I was confused.  You have structs defined in Go, and you need
 them to be defined in C.  For some reason I was thinking about this
 the other way around.

 Sure, something like what you described sounds reasonable.


 >> > Right now I just look at structs, but it would make sense to me
 that if
 >> > you
 >> > declare
 >> > type Foo int
 >> > and you use it as a result to a function we should see
 >> > typedef GoInt Foo;
 >> > in the header file.  Would that make sense to add too?
 >>
 >> I doubt it.  C typedefs are not Go typedefs.  Go's types are much
 >> stricter.  I expect that changing this would break currently working
 >> code.
 >
 >
 > Wouldn't this just be preserving the name of the type in the C code?
 > Whether you call it a GoInt or a Foo in this case should be the same
 - it's
 > just an alias in C.  Not sure I see how that would break anything.

 Same confusion on my part.  I'm not sure the typedef names help much
 but I suppose it is fine.

 Ian

>>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving email

Re: [go-nuts] cgo: support for exporting structs?

[Andy]

Here is the original CR:

  https://go-review.googlesource.com/#/c/14060/

Locally I'd fixed the tests and added one for the unnamed struct case Ian
pointed out - but I never checked this in (see attached diff).

So after this is brought in line with the current source I think it just
needs to be modified to handle unnamed structs.

---
Andy Maloney  //  https://asmaloney.com
twitter ~ @asmaloney 


On Sun, Oct 16, 2016 at 4:17 PM, Richard Wilkes  wrote:

> Yes, that or just a link to the CR would be good. Thanks!
>
> - Rich
>
>
>
> On Oct 16, 2016, at 1:15 PM, Andy  wrote:
>
> Ah - ok.
>
> As I recall, I submitted a CR for it but Ian pointed out an edge case I
> needed to handle. I never got to it because I changed direction and it
> didn't seem like anyone else was interested in this modification at the
> time.
>
> I can dig out the diffs from my previous attempt if that would help get
> you started - let me know.
>
> ---
> Andy Maloney  //  https://asmaloney.com
> twitter ~ @asmaloney 
>
>
> On Sun, Oct 16, 2016 at 3:07 PM, Richard Wilkes  wrote:
>
>> Hi, Andy.
>>
>> Thanks for replying. No, I was interested in the change to allow a struct
>> defined in Go to be passed to C, even if it only supported basic types.
>> Sounds like that part never made it in — certainly Go 1.7.1 doesn’t seem to
>> allow me to do such things, which is unfortunate.
>>
>> - Rich
>>
>>
>>
>> On Oct 16, 2016, at 10:23 AM, Andy  wrote:
>>
>> Richard:
>>
>> If you're asking about:
>>
>>   "cmd/cgo: annotate named return struct members in comments "
>>   https://go-review.googlesource.com/#/c/13061/
>>
>> It looks like that was merged at some point, but I'm not sure.
>>
>> I abandoned my experiments with go/cgo when some of the limitations
>> became clear.  I wouldn't ultimately be able to do what I'd wanted to with
>> it.
>>
>> ---
>> Andy Maloney  //  https://asmaloney.com
>> twitter ~ @asmaloney 
>>
>>
>> On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes 
>> wrote:
>>
>>> Andy,
>>>
>>> What's the status of getting this change into Go? I have an issue that
>>> could benefit from this particular support and was curious if/when I'd be
>>> able to use it.
>>>
>>> Thanks!
>>>
>>> - Rich
>>>
>>>
>>> On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:

 I imagine it will take some back-and-forth with you to get it into
 shape - make sure it's idiomatic and fits with the current structure - so
 I'll wait until after 1.5 is out.

 It requires changes I put in another CL (https://go-review.googlesourc
 e.com/#/c/13061/ ).  How do I handle that when I want to create a new
 CL?  Just branch off that one?  Or do just wait until the other one is
 accepted and merged?

 On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:
>
> On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney 
> wrote:
> > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor
> wrote:
> >>
> >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney 
> wrote:
> >> >
> >> > after all the Decls are added to the Package in Record() (in
> main.go),
> >> > call
> >> > a function which looks at each of the exported functions and uses
> the
> >> > Decls
> >> > to create a map to store some information about struct parameters
> &
> >> > results
> >> > in cgoType(), use the map that was just created to lookup the
> name of
> >> > the
> >> > struct and its size (which we calculated and stored in the map)
> and, if
> >> > we
> >> > find it, return it
> >> > at the end of writeExportHeader(), write out typedefs for each of
> the
> >> > structs in the map
> >>
> >> I'm not quite sure what you mean here.  It seems to me that you can
> >> treat any struct used by an exported function as though the Go code
> >> said "C.struct_foo".  That should define in Go like other cgo
> >> structs.
> >
> > I'm not seeing how to do this, but does that mean from my example
> above I'd
> > end up with this in the header:
> >
> > typedef struct {
> > GoIntr0;
> > GoStringr1;
> > } struct_Bar;
> >
> >
> > extern struct_Bar Foo();
> >
> > (And I'd lose the names of the members like I currently do in the
> return
> > structs?)
>
> Sorry, I was confused.  You have structs defined in Go, and you need
> them to be defined in C.  For some reason I was thinking about this
> the other way around.
>
> Sure, something like what you described sounds reasonable.
>
>
> >> > Right now I just look at structs, but it would make sense to me
> that if
> >> > you
> >> > declare
> >> > type Foo int
> >> > and you use it as a result to a function we should see
> >> > typedef GoInt Foo;
> >> > in the 

Re: [go-nuts] Re: Duplicate File Checker Performance

[Michael Jones]

Oh, I see. Well if you must read and hash every byte of every file then you 
really are mostly measuring device speed.

 

From:  on behalf of Sri G 

Date: Sunday, October 16, 2016 at 12:17 PM
To: golang-nuts 
Cc: 
Subject: Re: [go-nuts] Re: Duplicate File Checker Performance

 

This isn't exactly the same because I deleted some files but it shouldn't 
really matter.  

 

Switched to md5..

 

--- a/dup.go

+++ b/dup.go

@@ -3,7 +3,7 @@ package main

-   "crypto/sha256"

+   "crypto/md5"

 

@@ -207,8 +207,8 @@ func main() {

+type Hash [16]byte // appropriate for MD5

+// type Hash [32]byte // appropriate for SHA-256

 

 func hashFile(p string, hash []byte, prefix int64) (count int64) {

@@ -221,8 +221,8 @@ func hashFile(p string, hash []byte, prefix int64) (count 
int64) {

+   hasher := md5.New() // select MD5 in concert with "Hash" above

+   // hasher := sha256.New() // select SHA-256 in concert with "Hash" above

 

 

Checking only same sized files is huge speed up (82x less bytes checked)

 

016/10/16  14:33:51  total:  566 files ( 100.00%),1667774744 bytes 
( 100.00%)

2016/10/16 14:33:51   examined:9 files (   1.59%),  20271440 bytes 
(   1.22%) in 0.4209 seconds

2016/10/16 14:33:51 duplicates:9 files (   1.59%),  20271440 bytes 
(   1.22%)

 

Checking the first 4KB of files and only hashing if they are the same is 
another cool optimization (check avg. 768x less bytes in my case). Really nice 
Michael.

 

With workers = 8:

 

RAID10: 0.05s user 0.04s system 37% cpu 0.231 total, couldn't check memory 
usage but its probably negligible

SSD:0.05s user 0.04s system 59% cpu 0.137 total

 

Since SSD's and my filesystem are optimized for 4K random reads, it makes sense 
to use multiple threads/goroutines.

 

Optimal # of workers=9 on RAID 10: 0.05s user 0.04s system 40% cpu 0.220 total

on SSD workers = 8~9:  0.04s user 0.04s system 68% cpu 0.117 total

 

Not so much when you're doing a full sequential read. Because I use the md5 for 
other purposes, the entire file must be hashed, so sadly I cant use these 
optimizations.


On Sunday, October 16, 2016 at 1:26:24 PM UTC-4, Michael Jones wrote:

Sri G,

 

How does this time compare to my “Dup” program? I can’t test for you…since it 
is your filesystem…but I thought I had it going about as fast as possible a few 
years ago when I wrote that one.

 

https://github.com/MichaelTJones/dup

 

Michael

 

From:  on behalf of Sri G 
Date: Saturday, October 15, 2016 at 6:46 PM
To: golang-nuts 
Subject: [go-nuts] Re: Duplicate File Checker Performance

 

Thanks. Made the go code similar to python using CopyBuffer with a block size 
of 65536. 

 

buf := make([]byte, 65536)



if _, err := io.CopyBuffer(hash, file, buf); err != nil {

fmt.Println(err)

}

 

Didn't make too much of a difference, was slightly faster.

 

What got it to the same place was running ComputeHash in the same goroutine as 
the Walk function vs its own go routine for each file

 

+ComputeHash(path, info, queue, wg)

-go ComputeHash(path, info, queue, wg)

 

 

2.88s user 0.98s system 23% cpu 16.086 total, memory usage ~ 7MB

Here's the before and after pprof webs:

 

BEFORE with 'go ComputeHash(...):

 

 

 

AFTER with 'ComputeHash(...):

 

 

 

Since disk read are SOO much slower, computing the hash for each file in its 
own goroutine caused a huge slowdown.. 

 

btw this is on a RAID10, with SSD: 

 

Old code SSD: 3.31s user 17.87s system 244% cpu 8.667 total

 

New code SDD: 2.88s user 0.84s system 69% cpu 5.369 total

 

Shows you can throw hardware at a problem BUT the old code locks up my system 
momentarily..

 


On Saturday, October 15, 2016 at 3:27:38 PM UTC-4, Kevin Malachowski wrote:

Sorry, I meant that calling Write on the hash type might be slower if it's 
called more often.

(I'm on mobile right now. When I get back to a keyboard I'll try to come up 
with an example)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] cgo: support for exporting structs?

[Richard Wilkes]

Yes, that or just a link to the CR would be good. Thanks!

- Rich



> On Oct 16, 2016, at 1:15 PM, Andy  wrote:
> 
> Ah - ok.
> 
> As I recall, I submitted a CR for it but Ian pointed out an edge case I 
> needed to handle. I never got to it because I changed direction and it didn't 
> seem like anyone else was interested in this modification at the time.
> 
> I can dig out the diffs from my previous attempt if that would help get you 
> started - let me know.
> 
> ---
> Andy Maloney  //  https://asmaloney.com 
> twitter ~ @asmaloney 
> 
> On Sun, Oct 16, 2016 at 3:07 PM, Richard Wilkes  > wrote:
> Hi, Andy.
> 
> Thanks for replying. No, I was interested in the change to allow a struct 
> defined in Go to be passed to C, even if it only supported basic types. 
> Sounds like that part never made it in — certainly Go 1.7.1 doesn’t seem to 
> allow me to do such things, which is unfortunate.
> 
> - Rich
> 
> 
> 
>> On Oct 16, 2016, at 10:23 AM, Andy > > wrote:
>> 
>> Richard:
>> 
>> If you're asking about:
>> 
>>   "cmd/cgo: annotate named return struct members in comments "
>>   https://go-review.googlesource.com/#/c/13061/ 
>> 
>> 
>> It looks like that was merged at some point, but I'm not sure.
>> 
>> I abandoned my experiments with go/cgo when some of the limitations became 
>> clear.  I wouldn't ultimately be able to do what I'd wanted to with it.
>> 
>> ---
>> Andy Maloney  //  https://asmaloney.com 
>> twitter ~ @asmaloney 
>> 
>> On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes > > wrote:
>> Andy,
>> 
>> What's the status of getting this change into Go? I have an issue that could 
>> benefit from this particular support and was curious if/when I'd be able to 
>> use it.
>> 
>> Thanks!
>> 
>> - Rich
>> 
>> 
>> On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:
>> I imagine it will take some back-and-forth with you to get it into shape - 
>> make sure it's idiomatic and fits with the current structure - so I'll wait 
>> until after 1.5 is out.
>> 
>> It requires changes I put in another CL 
>> (https://go-review.googlesource.com/#/c/13061/ 
>>  ).  How do I handle that 
>> when I want to create a new CL?  Just branch off that one?  Or do just wait 
>> until the other one is accepted and merged?
>> 
>> On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:
>> On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney > wrote: 
>> > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor wrote: 
>> >> 
>> >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney > 
>> >> wrote: 
>> >> > 
>> >> > after all the Decls are added to the Package in Record() (in main.go), 
>> >> > call 
>> >> > a function which looks at each of the exported functions and uses the 
>> >> > Decls 
>> >> > to create a map to store some information about struct parameters & 
>> >> > results 
>> >> > in cgoType(), use the map that was just created to lookup the name of 
>> >> > the 
>> >> > struct and its size (which we calculated and stored in the map) and, if 
>> >> > we 
>> >> > find it, return it 
>> >> > at the end of writeExportHeader(), write out typedefs for each of the 
>> >> > structs in the map 
>> >> 
>> >> I'm not quite sure what you mean here.  It seems to me that you can 
>> >> treat any struct used by an exported function as though the Go code 
>> >> said "C.struct_foo".  That should define in Go like other cgo 
>> >> structs. 
>> > 
>> > I'm not seeing how to do this, but does that mean from my example above 
>> > I'd 
>> > end up with this in the header: 
>> > 
>> > typedef struct { 
>> > GoIntr0; 
>> > GoStringr1; 
>> > } struct_Bar; 
>> > 
>> > 
>> > extern struct_Bar Foo(); 
>> > 
>> > (And I'd lose the names of the members like I currently do in the return 
>> > structs?) 
>> 
>> Sorry, I was confused.  You have structs defined in Go, and you need 
>> them to be defined in C.  For some reason I was thinking about this 
>> the other way around. 
>> 
>> Sure, something like what you described sounds reasonable. 
>> 
>> 
>> >> > Right now I just look at structs, but it would make sense to me that if 
>> >> > you 
>> >> > declare 
>> >> > type Foo int 
>> >> > and you use it as a result to a function we should see 
>> >> > typedef GoInt Foo; 
>> >> > in the header file.  Would that make sense to add too? 
>> >> 
>> >> I doubt it.  C typedefs are not Go typedefs.  Go's types are much 
>> >> stricter.  I expect that changing this would break currently working 
>> >> code. 
>> > 
>> > 
>> > Wouldn't this just be preserving the name of the type in the C code? 
>> > Whether you call it a GoInt or a Foo in this case should be the same - 
>> > it's 
>> > just an alias in C.  Not sure I see how that would brea

[go-nuts] Centos - golang 1.6/1.7 - dial tcp i/o timeout

[subbagudipalli]

Getting dail tcp error while calling httpClient.Get or httpClient.Post


http client settings:
httpClient = &http.Client{
Timeout: 30 * time.Second,
}

Error:

dial tcp: lookup on xx.xx.xx.xxx:53: dial udp xx.xx.xx.xxx:53: i/o timeout


CentOS version: CentOS Linux release 7.2.1511 (Core) 


can you provide the resolution for this?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Liam]

A function to insert a byte into a slice is conspicuous by its absence.

Consider this code to parse /proc/meminfo into a 2D array and insert commas:

MemTotal: 497464 kB
MemFree:  386232 kB
MemAvailable: 452420 kB
Buffers:8612 kB
Cached:60160 kB
...
.
aLine := bytes.SplitN(fileBuf, []byte("\n"), 6)
var aPair [5][][]byte
for a := 0; a < 5; a++ {
  aPair[a] = bytes.SplitN(aLine[a], []byte(":"), 2)
  aPair[a][1] = sRegEx.FindSubmatch(aPair[a][1])[1] // " *([0-9]+)"
  if len(aPair[a][1]) > 3 {
// should be:
// aPair[a][1] = bytes.InsertByte(aPair[a][1], len(aPair[a][1])-3, 
',')

// but instead it's gobbledygook:
aPair[a][1] = aPair[a][1][:len(aPair[a][1])+1]
copy(aPair[a][1][len(aPair[a][1])-2:], 
aPair[a][1][len(aPair[a][1])-3:])
aPair[a][1][len(aPair[a][1])-4] = ','
  }
}

Surely this is common enough to merit inclusion in stdlib.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] cgo: support for exporting structs?

[Richard Wilkes]

Hi, Andy.

Thanks for replying. No, I was interested in the change to allow a struct 
defined in Go to be passed to C, even if it only supported basic types. Sounds 
like that part never made it in — certainly Go 1.7.1 doesn’t seem to allow me 
to do such things, which is unfortunate.

- Rich



> On Oct 16, 2016, at 10:23 AM, Andy  wrote:
> 
> Richard:
> 
> If you're asking about:
> 
>   "cmd/cgo: annotate named return struct members in comments "
>   https://go-review.googlesource.com/#/c/13061/ 
> 
> 
> It looks like that was merged at some point, but I'm not sure.
> 
> I abandoned my experiments with go/cgo when some of the limitations became 
> clear.  I wouldn't ultimately be able to do what I'd wanted to with it.
> 
> ---
> Andy Maloney  //  https://asmaloney.com 
> twitter ~ @asmaloney 
> 
> On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes  > wrote:
> Andy,
> 
> What's the status of getting this change into Go? I have an issue that could 
> benefit from this particular support and was curious if/when I'd be able to 
> use it.
> 
> Thanks!
> 
> - Rich
> 
> 
> On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:
> I imagine it will take some back-and-forth with you to get it into shape - 
> make sure it's idiomatic and fits with the current structure - so I'll wait 
> until after 1.5 is out.
> 
> It requires changes I put in another CL 
> (https://go-review.googlesource.com/#/c/13061/ 
>  ).  How do I handle that when 
> I want to create a new CL?  Just branch off that one?  Or do just wait until 
> the other one is accepted and merged?
> 
> On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:
> On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney > wrote: 
> > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor wrote: 
> >> 
> >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney > 
> >> wrote: 
> >> > 
> >> > after all the Decls are added to the Package in Record() (in main.go), 
> >> > call 
> >> > a function which looks at each of the exported functions and uses the 
> >> > Decls 
> >> > to create a map to store some information about struct parameters & 
> >> > results 
> >> > in cgoType(), use the map that was just created to lookup the name of 
> >> > the 
> >> > struct and its size (which we calculated and stored in the map) and, if 
> >> > we 
> >> > find it, return it 
> >> > at the end of writeExportHeader(), write out typedefs for each of the 
> >> > structs in the map 
> >> 
> >> I'm not quite sure what you mean here.  It seems to me that you can 
> >> treat any struct used by an exported function as though the Go code 
> >> said "C.struct_foo".  That should define in Go like other cgo 
> >> structs. 
> > 
> > I'm not seeing how to do this, but does that mean from my example above I'd 
> > end up with this in the header: 
> > 
> > typedef struct { 
> > GoIntr0; 
> > GoStringr1; 
> > } struct_Bar; 
> > 
> > 
> > extern struct_Bar Foo(); 
> > 
> > (And I'd lose the names of the members like I currently do in the return 
> > structs?) 
> 
> Sorry, I was confused.  You have structs defined in Go, and you need 
> them to be defined in C.  For some reason I was thinking about this 
> the other way around. 
> 
> Sure, something like what you described sounds reasonable. 
> 
> 
> >> > Right now I just look at structs, but it would make sense to me that if 
> >> > you 
> >> > declare 
> >> > type Foo int 
> >> > and you use it as a result to a function we should see 
> >> > typedef GoInt Foo; 
> >> > in the header file.  Would that make sense to add too? 
> >> 
> >> I doubt it.  C typedefs are not Go typedefs.  Go's types are much 
> >> stricter.  I expect that changing this would break currently working 
> >> code. 
> > 
> > 
> > Wouldn't this just be preserving the name of the type in the C code? 
> > Whether you call it a GoInt or a Foo in this case should be the same - it's 
> > just an alias in C.  Not sure I see how that would break anything. 
> 
> Same confusion on my part.  I'm not sure the typedef names help much 
> but I suppose it is fine. 
> 
> Ian 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] cgo: support for exporting structs?

[Richard Wilkes]

Andy,

What's the status of getting this change into Go? I have an issue that 
could benefit from this particular support and was curious if/when I'd be 
able to use it.

Thanks!

- Rich

On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:
>
> I imagine it will take some back-and-forth with you to get it into shape - 
> make sure it's idiomatic and fits with the current structure - so I'll wait 
> until after 1.5 is out.
>
> It requires changes I put in another CL (
> https://go-review.googlesource.com/#/c/13061/ ).  How do I handle that 
> when I want to create a new CL?  Just branch off that one?  Or do just wait 
> until the other one is accepted and merged?
>
> On Sunday, August 9, 2015 at 5:04:03 PM UTC-4, Ian Lance Taylor wrote:
>>
>> On Sun, Aug 9, 2015 at 1:49 PM, Andy Maloney  wrote: 
>> > On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor wrote: 
>> >> 
>> >> On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney  
>> wrote: 
>> >> > 
>> >> > after all the Decls are added to the Package in Record() (in 
>> main.go), 
>> >> > call 
>> >> > a function which looks at each of the exported functions and uses 
>> the 
>> >> > Decls 
>> >> > to create a map to store some information about struct parameters & 
>> >> > results 
>> >> > in cgoType(), use the map that was just created to lookup the name 
>> of 
>> >> > the 
>> >> > struct and its size (which we calculated and stored in the map) and, 
>> if 
>> >> > we 
>> >> > find it, return it 
>> >> > at the end of writeExportHeader(), write out typedefs for each of 
>> the 
>> >> > structs in the map 
>> >> 
>> >> I'm not quite sure what you mean here.  It seems to me that you can 
>> >> treat any struct used by an exported function as though the Go code 
>> >> said "C.struct_foo".  That should define in Go like other cgo 
>> >> structs. 
>> > 
>> > I'm not seeing how to do this, but does that mean from my example above 
>> I'd 
>> > end up with this in the header: 
>> > 
>> > typedef struct { 
>> > GoIntr0; 
>> > GoStringr1; 
>> > } struct_Bar; 
>> > 
>> > 
>> > extern struct_Bar Foo(); 
>> > 
>> > (And I'd lose the names of the members like I currently do in the 
>> return 
>> > structs?) 
>>
>> Sorry, I was confused.  You have structs defined in Go, and you need 
>> them to be defined in C.  For some reason I was thinking about this 
>> the other way around. 
>>
>> Sure, something like what you described sounds reasonable. 
>>
>>
>> >> > Right now I just look at structs, but it would make sense to me that 
>> if 
>> >> > you 
>> >> > declare 
>> >> > type Foo int 
>> >> > and you use it as a result to a function we should see 
>> >> > typedef GoInt Foo; 
>> >> > in the header file.  Would that make sense to add too? 
>> >> 
>> >> I doubt it.  C typedefs are not Go typedefs.  Go's types are much 
>> >> stricter.  I expect that changing this would break currently working 
>> >> code. 
>> > 
>> > 
>> > Wouldn't this just be preserving the name of the type in the C code? 
>> > Whether you call it a GoInt or a Foo in this case should be the same - 
>> it's 
>> > just an alias in C.  Not sure I see how that would break anything. 
>>
>> Same confusion on my part.  I'm not sure the typedef names help much 
>> but I suppose it is fine. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Serialization internal data to disk

[antoniosun001]

On Sunday, October 16, 2016 at 2:20:59 AM UTC-4, Tamás Gulácsi wrote:
>
> Please check your errors if you depend on the call's success! I.e. if you 
> write into a file, f.Close must be successful, or else maybe it doesn't 
> flush at all, and your file will be empty.


That's a good point, still not the true cause -- checking them will be a 
good practice, but will not solve the problem here, because the exact same 
code works in 
https://github.com/suntong/lang/blob/master/lang/Go/src/ds/PersistentData-GOB.go.
 
I've verify from the code and result that the save *is* actually working. 

Please take a look at PersistentData-GOB.go 
,
 
I assure you it is working. FYC, here is the excerpt:

func main() {
 test0()
 test22()
 test1()
 test2()
 test22()
 test22C()
}
...


 func test22C() {
 data := &Data{}
 RestoreState("data.gob", &data)
 defer SaveState("data.gob", &data)

 fmt.Printf("%#v\nSave it!\n", data)
 //time.Sleep(5 * time.Second)


 data = &Data{
 ID:  "226622",
 Payload: []byte("foobar"),
 Created: time.Now().Unix(),
 }
}


If you have run it, you'd know it is working. 
If you have looked at the code, you'd know that the error checking has been 
done, in "test2()". 

The reason that I didn't do it, is because I don't know how to do it. In 
essence, that "f.Close()" that you worried about is wrapped in "defer 
f.Close()" in SaveState, which in turn wrapped in "defer SaveState" in a 
function. I.e., all is happening during the program tear-down phase. If we 
are not planting a "log.Fatal(err)" bomb in SaveState itself, how do you 
think we should check it properly? 

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Problems with lack of mentioning of interfaces implemented by a concrete type

[Ali Demir]

Eric, imagine the following scenario:

You write a Java class that implements 2 different thirdparty interfaces 
that are very similar (Thus single impl). One of those interfaces resides 
in a jar that puts licensing limitations to your code/product. That becomes 
a problem. Go gives you the option to implement the interface without 
having the external dependency. Go code will still compile and run in 
environments where that 3rd party library that hosts the interface is NOT 
present. It will also support that 3rd party when it is present. But in 
Java all deployments must have that 3rd party jar and that can limit your 
commercial options. Working around this requires some acrobatics in Java 
(separate implementations). This is one of the reasons I like go.

Regards,
Ali


On Monday, March 29, 2010 at 12:13:37 PM UTC-7, Eric Hawthorne wrote:
>
> I really think in this discussion there is conflation of the
> straightjacket-of-single-inheritance problem with
> the explicitness of mention of inheritance issue.
>
> Java and all single inheritance languages imho are problematic because
> they
> force arbitrary choice of which properties of things are more
> important than
> other properties, because only one property dimension at a time can be
> the basis
> for the next level of class inheritance. That unnecessary choosing is
> damaging, because it will
> lead to unnatural deep hierarchies, and could lead to a combinatoric
> explosion of incompatible
> hierarchies that want to be doing/representing the same thing.
>
> The stated reason for limiting to single inheritance usually amounts
> to: "Understanding the
> forest of trees, or lattice, of more general mixin types available for
> creating or comprehending
> the true meaning and functionality of my object (or the aspects of my
> object) is just too complex
> for most programmers."
>
> What I would say is that, yes, programming by judicious borrowing of
> bits and pieces of meaning and functionality that have already been
> provided by others is
> going to be, initially, for you the programmer, more difficult. But
> the end result will be small snippets
> of much more baggage-free re-usable code for yourself and others to
> use in the future.
>
> The solution to the inherent complexity of programming in the middle
> of a very large and incremental
> mixin-type lattice is better code visualization tools, better class
> documentation standards including eiffel-like
> class-invariant and pre-post assertions, etc.
>
> The reason I think that explicit mention of which interfaces/mixins
> you are specializing/incorporating is
> important, is because being a correct implementor/specializer of a
> behaviourally defined type is much more
> involved than being method-signature compatible. You must be test-
> compatible (invariant-compatible,
> method-expected-behaviour-compatible), and header-comment compatible
> as well.
> You explicitly SHOULD NOT re-comment in your specialized type what it
> is that one of your more-general-interface-defined
> methods is doing. You should only comment about how it specializes the
> general behaviour. Why?
> Because comments are as important as code, and maintaining multiple
> versions of the same method header comment
> in different but related types is as bad as violating the DRY
> principle for code itself.
>
> EOR   (End Of Rant) - I know, I'm a hopeless idealist and incurably
> naive code romantic. I can always dream though...
>
>
> On Mar 28, 7:38 pm, Peter Froehlich 
> wrote:
> > Hi all,
> >
> > To paraphrase Knuth: "Premature standardization of interfaces is the
> > root of all evil." Some of this sounds like you have not yet lived
> > down the "more explicit please" phase we all go through. Not tying
> > implementation types to interface types in the language is a Good
> > Thing (tm) because it avoids one of the basic problems for
> > component-based software: That of *not* wanting to agree on One True
> > Interface (to rule them all). Once you get to the point where you have
> > to implement two distinct interfaces in one type but now the compiler
> > doesn't let you because they happen to disagree on some unimportant
> > method, you'll know the pain you've gotten yourself into. True, Go
> > already alleviates a little of this, but not enough to make it
> > advisable to put explicit declarations back. So please, whoever on the
> > Go team reads this thread, don't get tempted into the "but explicit
> > interface declarations are better" side of the force. Implicit
> > interface compatibility is the way!
> >
> > Cheers,
> > Peter
> > --
> > Peter H. Froehlich 
> > Senior Lecturer | Director, Johns Hopkins Gaming Lab
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/o

Re: [go-nuts] Is there a way to check if all pending finalizers have been executed?

['Kevin Malachowski' via golang-nuts]

In that case it's generally "better" to have an explicit Close on your Go type 
which causes the explicit freeing of C memory. This can be tedious depending on 
your specific code, though.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Ian Lance Taylor]

On Sun, Oct 16, 2016 at 1:49 PM, Liam  wrote:
>
> A function to insert a byte into a slice is conspicuous by its absence.

It's a one-liner.
append(s[:p], append([]byte{b}, s[p:]...)...)
(You may want to look at https://github.com/golang/go/wiki/SliceTricks).


> // but instead it's gobbledygook:

In particular cases where the one-liner is goobledygook, it's easy
enough to write a helper function yourself.

To argue that this should go into the standard library, look at some
corpus of Go code and find out how often it occurs.  If it occurs
fairly often, you've got a good case.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Centos - golang 1.6/1.7 - dial tcp i/o timeout

[Ian Lance Taylor]

On Sun, Oct 16, 2016 at 9:10 AM,   wrote:
>
> Getting dail tcp error while calling httpClient.Get or httpClient.Post
>
>
> http client settings:
> httpClient = &http.Client{
> Timeout: 30 * time.Second,
> }
>
> Error:
>
> dial tcp: lookup on xx.xx.xx.xxx:53: dial udp xx.xx.xx.xxx:53: i/o timeout
>
>
> CentOS version: CentOS Linux release 7.2.1511 (Core)
>
>
> can you provide the resolution for this?

Looks like your DNS server is not responding to your Go program.  Any
idea why?  Firewall issue?  Misunderstanding of DNS config files?  You
could try setting `GODEBUG=netdns=cgo` in the environment; see
https://golang.org/pkg/net.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a way to check if all pending finalizers have been executed?

[Ian Lance Taylor]

On Sat, Oct 15, 2016 at 12:08 AM,   wrote:
>
> On Saturday, October 15, 2016 at 8:18:04 AM UTC+8, Ian Lance Taylor wrote:
>>
>> On Fri, Oct 14, 2016 at 4:08 PM, 'Peter Lam' via golang-nuts
>>  wrote:
>> > Is there someway to wait for all pending finalizers to be run?
>>
>> Not in general, no.  Conceptually it doesn't make sense since, as you
>> know, finalizers not guaranteed to run at all.  You could of course
>> write your finalizers to support this.
>
> if finalizers not guaranteed to run at all, then what is its meaningfulness?

The current gc implementation has a concurrent GC.  A finalizer can
only run when the GC determines that the object is unreachable.  That
means that the question of when a finalizer runs is very nebulous.  If
the GC never completes--if it never determines that the objects is
unreachable--the finalizer will never run.  The GC only kicks in when
your program allocates enough new memory.  If your program never
allocates memory, the GC will never kick in, and the finalizer will
never run.  Many entirely reasonable Go programs do not allocate very
much memory.  Many of those programs use finalizers, because os.File
has a finalizer.  Those finalizers may never run.  They are still
meaningful, in that they will run if the program does allocate memory
and does complete a GC and does run finalizers.

To get back to your question, in the current gc implementation
finalizers are queued up to to be run by a goroutine.  That goroutine
simply runs each finalizer in order.  There is no way to wait for the
goroutine to complete.  In fact, it never completes; if there are no
finalizers to run, it goes to sleep waiting for the GC to release
another object with a finalizer.  There is also no way of telling
whether it has gone to sleep.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is there a way to check if all pending finalizers have been executed?

[Ian Lance Taylor]

On Sat, Oct 15, 2016 at 10:16 PM,   wrote:
>
> On Saturday, October 15, 2016 at 6:37:08 PM UTC+8, Hotei wrote:
>>
>> re "meaningfullness" - I think he's saying that a finalizer for a function
>> called in a goroutine might not run if main() quits first, intentionally or
>> otherwise.  You can of course check for this specific case by making sure
>> all your goroutines are cleaned up before exiting main - but in some (many?)
>> cases that's overkill.
>>
>>  If it's REALLY important to know which finalizer actions completed you
>> could log them to disk and analyse the results afterwards to see that all
>> the boxes got checked. Not quite what the OP was looking for I know - but
>> might help diagnose problems.
>
>
>
> Can you enum any use cases of SetFinalizer? I mean the use cases where
> alternative solutions would do worse than SetFinalizer.

SetFinalizer is useful when a Go value ties up resources that are not
themselves part of the Go program--they are part of some non-Go code
liked into the Go program, or they are some external resource.  If the
lifespan of the Go value is unclear--if it is passed around to several
different goroutines with no clear point where it is explicitly
released--then a finalizer can be a useful way to release the external
resource.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Liam]

On Sunday, October 16, 2016 at 2:56:42 PM UTC-7, Ian Lance Taylor wrote:
>
>  
> > A function to insert a byte into a slice is conspicuous by its absence. 
>
> It's a one-liner. 
> append(s[:p], append([]byte{b}, s[p:]...)...) 
>

This is already gobbledygook :) Does that read "Insert" to you? It doesn't 
to me.
 

> it's easy enough to write a helper function yourself. 
>

Of course, but two reasons for the stdlib are:
  foster semantic clarity in client code
  prevent innumerable variations across packages in the name and signature 
of a common function

To argue that this should go into the standard library, look at some 
> corpus of Go code and find out how often it occurs.  If it occurs 
> fairly often, you've got a good case. 
>

Fairly often; i.e. as often as Replace or Trim*? In my experience it passes 
that test.

That Insert and Delete are missing implies to me that there is something 
strange about such operations.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Ian Lance Taylor]

On Sun, Oct 16, 2016 at 3:34 PM, Liam  wrote:
>
> On Sunday, October 16, 2016 at 2:56:42 PM UTC-7, Ian Lance Taylor wrote:
>
>> To argue that this should go into the standard library, look at some
>> corpus of Go code and find out how often it occurs.  If it occurs
>> fairly often, you've got a good case.
>
>
> Fairly often; i.e. as often as Replace or Trim*? In my experience it passes
> that test.

This is something that can actually be measured.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Liam]

On Sunday, October 16, 2016 at 5:13:36 PM UTC-7, Ian Lance Taylor wrote:
>
> On Sun, Oct 16, 2016 at 3:34 PM, Liam > 
> wrote: 
> > 
> > On Sunday, October 16, 2016 at 2:56:42 PM UTC-7, Ian Lance Taylor wrote: 
> > 
> >> To argue that this should go into the standard library, look at some 
> >> corpus of Go code and find out how often it occurs.  If it occurs 
> >> fairly often, you've got a good case. 
> > 
> > 
> > Fairly often; i.e. as often as Replace or Trim*? In my experience it 
> passes 
> > that test. 
>
> This is something that can actually be measured. 
>

By doing grep 'append.*append' ? And '\[:.*len\(.*\).*\+[0-9]+.*\]' ?
Neither of these will be accurate.

Part of my point is that Insert operations are semantically obscured.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Ian Lance Taylor]

On Sun, Oct 16, 2016 at 5:25 PM, Liam  wrote:
>
> On Sunday, October 16, 2016 at 5:13:36 PM UTC-7, Ian Lance Taylor wrote:
>>
>> On Sun, Oct 16, 2016 at 3:34 PM, Liam  wrote:
>> >
>> > On Sunday, October 16, 2016 at 2:56:42 PM UTC-7, Ian Lance Taylor wrote:
>> >
>> >> To argue that this should go into the standard library, look at some
>> >> corpus of Go code and find out how often it occurs.  If it occurs
>> >> fairly often, you've got a good case.
>> >
>> >
>> > Fairly often; i.e. as often as Replace or Trim*? In my experience it
>> > passes
>> > that test.
>>
>> This is something that can actually be measured.
>
>
> By doing grep 'append.*append' ? And '\[:.*len\(.*\).*\+[0-9]+.*\]' ?
> Neither of these will be accurate.
>
> Part of my point is that Insert operations are semantically obscured.

Understood.  But my guess is that insertion operations are fairly
rare--much less common than Replace or Trim.  People obviously append
to slices all the time, but I suspect that they tend to write their
slice operations to not require inserting a single element in the
middle of the slice, because insertion is by definition O(N) rather
than O(1).  I could certainly be wrong, but that guess leads me to
suspect that adding a function for something that can easily be
written in a single expression is overkill.  Since I am only guessing,
and since it seems to me that you are also guessing, I think we should
measure rather than guess.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Why no bytes.InsertByte(s []byte, p int, b byte) []byte ?

[Liam Breck]

On Sun, Oct 16, 2016 at 5:32 PM, Ian Lance Taylor  wrote:

> On Sun, Oct 16, 2016 at 5:25 PM, Liam  wrote:
> >
> > On Sunday, October 16, 2016 at 5:13:36 PM UTC-7, Ian Lance Taylor wrote:
> >>
> >> On Sun, Oct 16, 2016 at 3:34 PM, Liam  wrote:
> >> >
> >> > On Sunday, October 16, 2016 at 2:56:42 PM UTC-7, Ian Lance Taylor
> wrote:
> >> >
> >> >> To argue that this should go into the standard library, look at some
> >> >> corpus of Go code and find out how often it occurs.  If it occurs
> >> >> fairly often, you've got a good case.
> >> >
> >> >
> >> > Fairly often; i.e. as often as Replace or Trim*? In my experience it
> >> > passes
> >> > that test.
> >>
> >> This is something that can actually be measured.
> >
> >
> > By doing grep 'append.*append' ? And '\[:.*len\(.*\).*\+[0-9]+.*\]' ?
> > Neither of these will be accurate.
> >
> > Part of my point is that Insert operations are semantically obscured.
>
> Understood.  But my guess is that insertion operations are fairly
> rare--much less common than Replace or Trim.  People obviously append
> to slices all the time, but I suspect that they tend to write their
> slice operations to not require inserting a single element in the
> middle of the slice, because insertion is by definition O(N) rather
> than O(1).  I could certainly be wrong, but that guess leads me to
> suspect that adding a function for something that can easily be
> written in a single expression is overkill.  Since I am only guessing,
> and since it seems to me that you are also guessing, I think we should
> measure rather than guess.
>

You characterize optimization as a widespread habit. Most code isn't
optimized, as it doesn't need to be. Most coders follow the first advice
found, which in my case was Insert() on [1]. If you wish the most optimal
solution to be widespread, stdlib would be a good vehicle :-)

I would attempt to provide data confirming my guess, but I can't see how to
do that without spending a lot of time hunting for semantically invisible
operations. Since there are two authoritative suggestions for this op, it's
a reasonable guess that it's widely done.

BTW, we are having this discussion because fmt does not support thousands
grouping, which also seems like a common need.

[1] https://blog.golang.org/slices

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Starting detached child process

[me]

For some reason, this does not seem to work for `ssh` (at least on OS X) 
unless you have `ControlPath` in ssh config. Are there any side-effects of 
SysProcAttr that I could be missing?

On Friday, July 18, 2014 at 1:07:44 AM UTC-7, Ian Lance Taylor wrote:
>
> On Thu, Jul 17, 2014 at 8:12 PM, Hein Meling  > wrote: 
> > 
> > Your solution seems to work when then parent process exits cleanly. But 
> if 
> > you do: 
> > 
> > func main() { 
> > cmd := exec.Command("sleep", "15") 
> > cmd.Start() 
> > time.Sleep(20 * time.Second) 
> > } 
> > 
> > And CTRL-C the parent process before it exits cleanly, then also the 
> sleep 
> > process goes away. Does the same happen to your monkey.sh process? 
> > 
> > Moreover, my use case is actually somewhat more involved. That is, I 
> would 
> > like to do cmd.Wait() (from different goroutines), in order to monitor 
> the 
> > child processes, but I don't want the child processes to fail if the 
> parent 
> > fails. 
>
> ^C kills all the processes because on Unix typing ^C on the terminal 
> sends a signal to every process in the process group.  So you want to 
> have a child process that is in a different process group.  You do 
> that by setting Setpgid to true in the Cmd.SysProcAttr field. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How to setup go-swagger for an existing project to generate API docs

[SakShak]

I have an existing project that needs a UI for the API. I want to use go 
swagger but I am completely confused
https://github.com/go-swagger/go-swagger/tree/master/examples/todo-list

I want to set it up so I add annotations in the code and then run the 
command swagger generate spec and it would generate the spec

However whenever I run it, it prints 
{"swagger":"2.0","paths":{},"definitions":{}}

My project structure is as follows

project/ 
  main.go
  api/
router.go

In main.go I have this annotation

//go:generate swagger generate spec 
package main

In router above one of my handlers I have this annotation

// swagger:route GET /profile
//
// Gets profile of user
//
// Produces:
// - application/json
// - application/x-protobuf
//
// Schemes: http, https, ws, wss
//
// Security:
//   api_key:
//   oauth: read, write
//
// Responses:
//   default: genericError
//   200: someResponse
//   422: validationError
r.GET("/profile", profileHandler

I've been stuck trying to set up an api generator for a while. Any help is 
much appreciated. If you have experience setting it up, please let me know 
how you did it

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Serialization internal data to disk

[Tamás Gulácsi]

2016. október 16., vasárnap 22:55:24 UTC+2 időpontban antoni...@gmail.com a 
következőt írta:
>
>
>
> On Sunday, October 16, 2016 at 2:20:59 AM UTC-4, Tamás Gulácsi wrote:
>>
>> Please check your errors if you depend on the call's success! I.e. if you 
>> write into a file, f.Close must be successful, or else maybe it doesn't 
>> flush at all, and your file will be empty.
>
>
> That's a good point, still not the true cause -- checking them will be a 
> good practice, but will not solve the problem here, because the exact same 
> code works in 
> https://github.com/suntong/lang/blob/master/lang/Go/src/ds/PersistentData-GOB.go.
>  
> I've verify from the code and result that the save *is* actually working. 
>
> ...
>
> If you have run it, you'd know it is working. 
> If you have looked at the code, you'd know that the error checking has 
> been done, in "test2()". 
>
>
Yes, you're right: I haven't tried or read your code, as the "check your 
errors" is not just a basic advice, it's a MUST!
We even have a tool to check that you check all your errors: 
github.com/kisielk/errchek :)

And that your unit tests pass means only one thing: your unit tests pass. 
It does NOT mean that your production code is correct!
Maybe your unit tests does not reveal the error, as your unit tests does 
sth else then the production code.
Maybe different environment, different file permissions, or even different 
code paths...
 

> The reason that I didn't do it, is because I don't know how to do it. In 
> essence, that "f.Close()" that you worried about is wrapped in "defer 
> f.Close()" in SaveState, which in turn wrapped in "defer SaveState" in a 
> function. I.e., all is happening during the program tear-down phase. If we 
> are not planting a "log.Fatal(err)" bomb in SaveState itself, how do you 
> think we should check it properly? 
>
>
There are several possibilities. Two easy:
1. use a named return parameter:
func SaveState(...) (err error) {
...
// close the file and return the error
defer func() { 
  if closeErr := f.Close(); closeErr != nil && err == nil {
err = closeErr
}()
...
}

2. Treat the defer f.Close() just as a resource releaser for error path, 
and do an "return f.Close()" anyway:
func SaveState(...) error {
...
defer f.Close() // just release resources on error paths
...
return f.Close() // never lose the error of Close!
}

yes, this does call f.Close() twice, and the second (the defer) will error 
out, but that does no harm.

> Thanks
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.