[go-nuts] Go as your first language

[James Pettyjohn]

I've had multiple occasions where I've needed to train someone to be a 
programmer from scratch in a Go environment.

Trouble I've found is while the go texts are simple and straightforward, 
relatively speaking, they often written by someone who sought a better life 
in go, fleeing Java/C/C++. They will routinely reference these other 
languages in examples, touting the benefits of go is comparison to the old 
language. Much like reading GOF design patterns without a background in 
smalltalk, it is hard for new developers to pick up when they don't know 
other languages first. Commonly they cut it back and learn JS first.

Assuming they eventually picked up the language they now need to learn how 
to be a software engineer and write code that doesn't suck. Especially 
present with those who just learned how to program using JS. And what I've 
seen on the subject often expects a knowledge of another language.

Are there tracks of knowledge to take someone from 0 to understanding 
baseline knowledge?  

And from there through taking them to a professional grade standard?

Best,
James

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] how about golang channel memory usage?

[Lynn H]

OK, get it ,
you means virt memory use, I test it, result data could match the table

but how about res memory usage? how  RES memory used in golang? 

在 2018年1月16日星期二 UTC+8下午2:23:08，Jan Mercl写道：
>
> On Tue, Jan 16, 2018 at 4:49 AM Lynn H  
> wrote:
>
> > not understand ur table,
> > when memsize is 256,single channel use 6114Byte memory?
>
> Size of []byte is 3 words, assuming a 64 bit system that's 24 bytes. 
> 256*24 = 6,144 bytes ie. size of a chan []byte with capacity 256, which is 
> what make(chan []byte, 256) produces.
>
> In general, make(chan T, N) means reserve N*sizeof(T) bytes for the 
> channel buffer.
>
>
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] how about golang channel memory usage?

[Jan Mercl]

On Tue, Jan 16, 2018 at 4:49 AM Lynn H  wrote:

> not understand ur table,
> when memsize is 256,single channel use 6114Byte memory?

Size of []byte is 3 words, assuming a 64 bit system that's 24 bytes. 256*24
= 6,144 bytes ie. size of a chan []byte with capacity 256, which is what
make(chan []byte, 256) produces.

In general, make(chan T, N) means reserve N*sizeof(T) bytes for the channel
buffer.



-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Experience Report building OCR in Go

[Kevin Malachowski]

Not sure about the GUI points: exp/shiny has worked really well for me and has 
existed for at least a year. I do admit that 4 years ago I tried my hand at a 
GUI framework for Go though...

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] how about golang channel memory usage?

[Lynn H]

not understand ur table,
when memsize is 256,single channel use  6114Byte memory?

在 2018年1月16日星期二 UTC+8上午1:46:45，Jan Mercl写道：
>
>
>
>
> On Mon, Jan 15, 2018 at 6:04 PM Lynn H  
> wrote:
>
> > so what the rules of channel size and memory usage? 
>
> Check [0] if the formulas are correct.
>
>   [0]: 
> https://docs.google.com/spreadsheets/d/1BuAnHmBKWfWD9JXXi4_k9ck-rkJyDuQgCQ05s4ytJI8/edit?usp=sharing
>
>
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: why can't change map when in for range statements?

[Kevin Powick]

I believe that the *delete()* function doesn't actually remove items from 
the map, only sets them to be ignored.  i.e.  The map data remains, but it 
is skipped during the range iteration.  Regardless, I think you need to 
reconsider the logic of your approach to exiting the loop in the first 
place.  Some type of a conditional with a *break* statement would seem 
better to me.

--
Kevin Powick

On Monday, 15 January 2018 22:00:30 UTC-5, sheepbao wrote:
>
> Thanks reply, I know *range expression is evaluated once before beginning 
> the loop, *but I delete map in for statements also really affected the 
> loop. why not set to nil can't affect real map.
>
> func mapTest() {
>
> m := map[int]int{1: 1, 2: 2}
>
> for k, v := range m {
>
> println(k, v)
>
> delete(m, 2)
>
> // m = nil
>
> }
>
> }
> // output:
> // 1 1
>
>
>
>
> On Tuesday, January 16, 2018 at 10:48:45 AM UTC+8, Kevin Powick wrote:
>>
>>
>>
>> On Monday, 15 January 2018 21:23:40 UTC-5, sheepbao wrote:
>>>
>>> I wrote this code, and why `map=nil` don't stop the loop?
>>> ```go
>>>
>>> func mapTest() {
>>>
>>> m := map[int]int{1: 1, 2: 2}
>>>
>>> for k, v := range m {
>>>
>>> println(k, v)
>>>
>>> m = nil
>>>
>>> }
>>>
>>> }
>>>
>>> ```
>>>
>>> output:
>>>
>>> 1 1
>>>
>>> 2 2
>>>
>>>
>>> I don't understand when I set  `m = nil`, the loop is not stop. m 
>>> doesn't seem to be affected.
>>>
>>
>> https://golang.org/ref/spec#For_statements
>>
>>
>> *The range expression is evaluated once before beginning the loop, with 
>> one exception: if the range expression is an array or a pointer to an array 
>> and at most one iteration variable is present, only the range expression's 
>> length is evaluated; if that length is constant, by definition the range 
>> expression itself will not be evaluated. *
>>
>>
>>
>> --
>> Kevin Powick
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: why can't change map when in for range statements?

[sheepbao]

Thanks reply, I know *range expression is evaluated once before beginning 
the loop, *but I delete map in for statements also really affected the 
loop. why not set to nil can't affect real map.

func mapTest() {

m := map[int]int{1: 1, 2: 2}

for k, v := range m {

println(k, v)

delete(m, 2)

// m = nil

}

}
// output:
// 1 1




On Tuesday, January 16, 2018 at 10:48:45 AM UTC+8, Kevin Powick wrote:
>
>
>
> On Monday, 15 January 2018 21:23:40 UTC-5, sheepbao wrote:
>>
>> I wrote this code, and why `map=nil` don't stop the loop?
>> ```go
>>
>> func mapTest() {
>>
>> m := map[int]int{1: 1, 2: 2}
>>
>> for k, v := range m {
>>
>> println(k, v)
>>
>> m = nil
>>
>> }
>>
>> }
>>
>> ```
>>
>> output:
>>
>> 1 1
>>
>> 2 2
>>
>>
>> I don't understand when I set  `m = nil`, the loop is not stop. m doesn't 
>> seem to be affected.
>>
>
> https://golang.org/ref/spec#For_statements
>
>
> *The range expression is evaluated once before beginning the loop, with 
> one exception: if the range expression is an array or a pointer to an array 
> and at most one iteration variable is present, only the range expression's 
> length is evaluated; if that length is constant, by definition the range 
> expression itself will not be evaluated. *
>
>
>
> --
> Kevin Powick
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: why can't change map when in for range statements?

[Kevin Powick]

On Monday, 15 January 2018 21:23:40 UTC-5, sheepbao wrote:
>
> I wrote this code, and why `map=nil` don't stop the loop?
> ```go
>
> func mapTest() {
>
> m := map[int]int{1: 1, 2: 2}
>
> for k, v := range m {
>
> println(k, v)
>
> m = nil
>
> }
>
> }
>
> ```
>
> output:
>
> 1 1
>
> 2 2
>
>
> I don't understand when I set  `m = nil`, the loop is not stop. m doesn't 
> seem to be affected.
>

https://golang.org/ref/spec#For_statements


*The range expression is evaluated once before beginning the loop, with one 
exception: if the range expression is an array or a pointer to an array and 
at most one iteration variable is present, only the range expression's 
length is evaluated; if that length is constant, by definition the range 
expression itself will not be evaluated. *



--
Kevin Powick


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] why can't change map when in for range statements?

[sheepbao]

I wrote this code, and why `map=nil` don't stop the loop?
```go

func mapTest() {

m := map[int]int{1: 1, 2: 2}

for k, v := range m {

println(k, v)

m = nil

}

}

```

output:

1 1

2 2


I don't understand when I set  `m = nil`, the loop is not stop. m doesn't 
seem to be affected.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] The side effect of calling html.Token()

[Tong Sun]

Thanks a lot for the comprehensive explanation Nigel, really appreciate it
!!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] SIGSEGV during build of Go from source

[James Waldrop]

Just to confirm, this indeed fixed things. Thanks for the quick response!

On Mon, Jan 15, 2018 at 2:31 PM, James Waldrop  wrote:

> Aha, I did. I was cross compiling. Thank you!
>
> On Mon, Jan 15, 2018 at 2:27 PM Ian Lance Taylor  wrote:
>
>> On Mon, Jan 15, 2018 at 1:43 PM,   wrote:
>> >
>> > My bootstrap Go is a 1.9.2 binary, and I'm trying to compile Go on High
>> > Sierra (MacOS 10.13.2).
>> >
>> > I get a SIGSEGV during test run:
>> >
>> > # cmd/go terminal test
>> > panic: runtime error: invalid memory address or nil pointer dereference
>> > [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x110ed55]
>> >
>> > goroutine 1 [running]:
>> > main.(*tester).registerTests.func2(0xc4201cc450, 0xc4201cc450, 0x14)
>> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:364 +0x45
>> > main.(*tester).run(0xc4201d2000)
>> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:204 +0x94e
>> > main.cmdtest()
>> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:42 +0x2b8
>> > main.xmain()
>> > /Users/jwaldrop/Projects/go/src/cmd/dist/main.go:43 +0x171
>> > main.main()
>> > /Users/jwaldrop/Projects/go/src/cmd/dist/util.go:509 +0x261
>> >
>> >
>> > I looked through issues on Github and did a quick search here and
>> didn't see
>> > anyone else who's had this problem. I can't repro just yet, but will be
>> able
>> > to try in a few hours. Meanwhile, here's what I did to get here
>> > (irrelevancies deleted):
>> >
>> >   606  git clone https://go.googlesource.com/go
>> >   607  cd go
>> >   611  git checkout go1.9.2
>> >   612  cd src
>> >   624  ./all.bash
>> >   625  which go
>> >   626  export GOROOT_BOOTSTRAP=/usr/local/go
>> >   627  ./all.bash
>> >   645  history
>> >
>> >
>> > My pre-built Go binary comes via brew.
>> >
>> > Any help is appreciated, I'm trying to speed up Linux cross compiles
>> without
>> > giving myself write access to /usr/local.
>>
>> The failing test, "cmd/go terminal test", is only run on GNU/Linux,
>> but you say that you are running on Darwin.  I don't understand why
>> that test is being run at all.  Do you have GOOS=linux set in the
>> environment?
>>
>> Ian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] therecipe/qt error from uitools

[rob]

Hi.  I'm interested in learning about therecipe/qt.

I followed the instructions for the docker image.  After running 
qtsetup, I got an error that uitools failed to install.


I don't know where to start, or if it even matters?

Thx


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] The side effect of calling html.Token()

[Nigel Tao]

On Sun, Jan 14, 2018 at 4:33 PM, Tong Sun  wrote:
> Not being able to do that, I have to save all the Token() info to different
> variables, then pass all those variables to my function separately, instead
> of passing merely a single tokenizer.

Instead of using different variables, I'd just pass the Token itself
around. It should already contain everything you need. For example, if
t is a variable of type html.Token, and t.Type is html.StartTagToken,
html.EndTagToken or html.SelfClosingTagToken, then t.Data is the tag
name, such as "script". It's a string-typed field, not a method that
returns a string, so there's no restrictions like those on calling
Tokenizer method multiple times.

As an optional, advanced level comment, t.DataAtom will also be a
hashed uint32 value of that string, for well known strings. For
example, the uint32 constant atom.Script (from the
golang.org/x/net/html/atom package) corresponds to a "script" tag.
Comparing uint32 values is noticably faster than comparing string
values, if you're doing a *lot* of tag name comparisons. For example,
I can't remember the exact number, but IIRC, the x/net/html parser
(which builds a DOM tree from the token stream) got a 10% or 30% speed
boost by comparing atoms instead of strings.


On Sun, Jan 14, 2018 at 4:53 PM, Tong Sun  wrote:
>  Actually, found out I only called Token() once:
>
> https://play.golang.org/p/HtevQ3RbQsi
>
>  reader := strings.NewReader("SomeText")
>  tokenizer := html.NewTokenizer(reader)
>  tokenizer.Next()
>  fmt.Println(tokenizer.TagName())
>  fmt.Println(tokenizer.Token())

Again, from the "EBNF" in the package documentation:


In EBNF notation, the valid call sequence per token is:

Next {Raw} [ Token | Text | TagName {TagAttr} ]


If you're not familiar with EBNF
(https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form),
this means that you can call the Token method (once), or call the
TagName method (once), but not call both.


> I.e., what I used in the loop the standard way was `TagName()`:
>
>  case html.StartTagToken, html.EndTagToken:
>  tn, _ := z.TagName()
>  tag := strings.ToLower(string(tn))

The strings.ToLower call should be unnecessary. As
https://godoc.org/golang.org/x/net/html#Tokenizer.TagName says,
"TagName returns the *lower-cased* name of a tag token", (emphasis
added).


> But by the time I need to call Token() within my function (printElmt) to get
> the full token info, it's already impossible.

As I said earlier in this message, just call Token (the method) once,
at the top of the loop, and switch on its Type field, pass the Token
(the type) around, etc.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Measuring low latency operations

[matthewjuran]

Another benchmarking rule is the operating system may be dynamically 
adjusting performance for power saving or other reasons. Be sure to disable 
any such features before running the benchmark.

Matt

On Monday, January 15, 2018 at 9:04:11 AM UTC-6, Jesper Louis Andersen 
wrote:
>
> General rules:
>
> * If possible, use the benchmark system inside the "testing" package. It 
> will eliminate some common mistakes in benchmarking, but keep your head 
> straight while doing it, so you don't accidentally measure setup time, or 
> something else which is outside the thing you are trying to test.
>
> * A fast benchmark is sensitive to variance on the machine. The "testing" 
> packages smooths it out by running the test more than once, but another 
> trick is to compute outlier variance through bootstrapping methods and 
> figure out if the result is likely to be skewed by outliers.
>
> * Likely, variance can occur from multiple sources:
>   - Servers which are virtualized compete with resources from neighboring 
> systems.
>   - Desktop computers can have your benchmark compete with e.g., your 
> music player or browsing while tests are being run.
>
> * When you report numbers, beware of the average. At least report the 
> median, and if it differs from the average, you should be worried since the 
> data set is unlikely to be normally distributed.
>
> * Prefer visual methods: Compute a kernel density plot of the data to see 
> how it distributes. You want to look out for odd shapes.
>
> * If your data sets have the same variance, are following a normal 
> distribution and are independent, you can compare them with e.g., 
> student's-t. FreeBSD has the ministat(1) tool by Poul-Henning Kamp for 
> this, and I know it is easily ported. But a bit of R or Python also does 
> the trick.
>
>
>
> On Mon, Jan 15, 2018 at 3:07 AM  wrote:
>
>> All:
>>
>> I am testing an application that has a latency of the order of few to 
>> several microseconds.  I wrote a benchmark and measured the performance.  I 
>> also computed the elapsed time using time.Now() and time.Since().  (I will 
>> change this to time,Nanoseconds and try shortly.
>>
>> I found the computed latencies had a large variation.
>>
>> Set size: 22024 
>>
>> Min: 20, Max: 936 Average: 32 SD: 43 all in microseconds 
>>
>>
>>  It's skewed towards the lower end of the spectrum.  go test itself 
>> reported 56 microseconds as the response time.
>>
>>1. Is this variation due to garbage collection?
>>1. If so, can the variation be somehow reduced by tuning the garbage 
>>   collector?
>>
>>
>> I will appreciate any references or thoughts.
>>
>> With regards,
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] SIGSEGV during build of Go from source

[James Waldrop]

Aha, I did. I was cross compiling. Thank you!

On Mon, Jan 15, 2018 at 2:27 PM Ian Lance Taylor  wrote:

> On Mon, Jan 15, 2018 at 1:43 PM,   wrote:
> >
> > My bootstrap Go is a 1.9.2 binary, and I'm trying to compile Go on High
> > Sierra (MacOS 10.13.2).
> >
> > I get a SIGSEGV during test run:
> >
> > # cmd/go terminal test
> > panic: runtime error: invalid memory address or nil pointer dereference
> > [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x110ed55]
> >
> > goroutine 1 [running]:
> > main.(*tester).registerTests.func2(0xc4201cc450, 0xc4201cc450, 0x14)
> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:364 +0x45
> > main.(*tester).run(0xc4201d2000)
> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:204 +0x94e
> > main.cmdtest()
> > /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:42 +0x2b8
> > main.xmain()
> > /Users/jwaldrop/Projects/go/src/cmd/dist/main.go:43 +0x171
> > main.main()
> > /Users/jwaldrop/Projects/go/src/cmd/dist/util.go:509 +0x261
> >
> >
> > I looked through issues on Github and did a quick search here and didn't
> see
> > anyone else who's had this problem. I can't repro just yet, but will be
> able
> > to try in a few hours. Meanwhile, here's what I did to get here
> > (irrelevancies deleted):
> >
> >   606  git clone https://go.googlesource.com/go
> >   607  cd go
> >   611  git checkout go1.9.2
> >   612  cd src
> >   624  ./all.bash
> >   625  which go
> >   626  export GOROOT_BOOTSTRAP=/usr/local/go
> >   627  ./all.bash
> >   645  history
> >
> >
> > My pre-built Go binary comes via brew.
> >
> > Any help is appreciated, I'm trying to speed up Linux cross compiles
> without
> > giving myself write access to /usr/local.
>
> The failing test, "cmd/go terminal test", is only run on GNU/Linux,
> but you say that you are running on Darwin.  I don't understand why
> that test is being run at all.  Do you have GOOS=linux set in the
> environment?
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] SIGSEGV during build of Go from source

[Ian Lance Taylor]

On Mon, Jan 15, 2018 at 1:43 PM,   wrote:
>
> My bootstrap Go is a 1.9.2 binary, and I'm trying to compile Go on High
> Sierra (MacOS 10.13.2).
>
> I get a SIGSEGV during test run:
>
> # cmd/go terminal test
> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x110ed55]
>
> goroutine 1 [running]:
> main.(*tester).registerTests.func2(0xc4201cc450, 0xc4201cc450, 0x14)
> /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:364 +0x45
> main.(*tester).run(0xc4201d2000)
> /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:204 +0x94e
> main.cmdtest()
> /Users/jwaldrop/Projects/go/src/cmd/dist/test.go:42 +0x2b8
> main.xmain()
> /Users/jwaldrop/Projects/go/src/cmd/dist/main.go:43 +0x171
> main.main()
> /Users/jwaldrop/Projects/go/src/cmd/dist/util.go:509 +0x261
>
>
> I looked through issues on Github and did a quick search here and didn't see
> anyone else who's had this problem. I can't repro just yet, but will be able
> to try in a few hours. Meanwhile, here's what I did to get here
> (irrelevancies deleted):
>
>   606  git clone https://go.googlesource.com/go
>   607  cd go
>   611  git checkout go1.9.2
>   612  cd src
>   624  ./all.bash
>   625  which go
>   626  export GOROOT_BOOTSTRAP=/usr/local/go
>   627  ./all.bash
>   645  history
>
>
> My pre-built Go binary comes via brew.
>
> Any help is appreciated, I'm trying to speed up Linux cross compiles without
> giving myself write access to /usr/local.

The failing test, "cmd/go terminal test", is only run on GNU/Linux,
but you say that you are running on Darwin.  I don't understand why
that test is being run at all.  Do you have GOOS=linux set in the
environment?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] SIGSEGV during build of Go from source

[jwaldrop]

My bootstrap Go is a 1.9.2 binary, and I'm trying to compile Go on High 
Sierra (MacOS 10.13.2).

I get a SIGSEGV during test run:

# cmd/go terminal test
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x110ed55]

goroutine 1 [running]:
main.(*tester).registerTests.func2(0xc4201cc450, 0xc4201cc450, 0x14)
/Users/jwaldrop/Projects/go/src/cmd/dist/test.go:364 +0x45
main.(*tester).run(0xc4201d2000)
/Users/jwaldrop/Projects/go/src/cmd/dist/test.go:204 +0x94e
main.cmdtest()
/Users/jwaldrop/Projects/go/src/cmd/dist/test.go:42 +0x2b8
main.xmain()
/Users/jwaldrop/Projects/go/src/cmd/dist/main.go:43 +0x171
main.main()
/Users/jwaldrop/Projects/go/src/cmd/dist/util.go:509 +0x261


I looked through issues on Github and did a quick search here and didn't 
see anyone else who's had this problem. I can't repro just yet, but will be 
able to try in a few hours. Meanwhile, here's what I did to get here 
(irrelevancies deleted):

  606  git clone https://go.googlesource.com/go
  607  cd go
  611  git checkout go1.9.2
  612  cd src
  624  ./all.bash
  625  which go
  626  export GOROOT_BOOTSTRAP=/usr/local/go
  627  ./all.bash
  645  history 


My pre-built Go binary comes via brew.

Any help is appreciated, I'm trying to speed up Linux cross compiles 
without giving myself write access to /usr/local.

James

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: double package vendor issue

[liming . jiang02]

Thanks. This really helps in solving my problem.

On Saturday, 27 February 2016 07:22:34 UTC+11, Nate Brennand wrote:
>
> A solution we use is simply removing all nested vendor directories:
> find vendor/ -path '*/vendor' -type d | xargs -IX rm -r X 
> 
> ​
>
> On Fri, Feb 26, 2016 at 12:19 PM,  
> wrote:
>
>> Maybe similar to https://github.com/golang/go/issues/12432
>> I understand that you can consolidate in your own repo into a single 
>> vendor but that is not the case here.
>>
>>
>> On Friday, February 26, 2016 at 12:10:49 PM UTC-8, jonathan...@gmail.com 
>> wrote:
>>>
>>> I have one repo with a sub package library like /reporoot/A/A.go
>>>
>>> Inside it uses the ssh package. It also exposes the ssh package types in 
>>> a few of its exported interfaces.
>>>
>>> So in another repo I have a command that imports reporoot/A. It must 
>>> also import the ssh package since it directly references the identifiers to 
>>> do with the export types from A. So it has ssh vendored.
>>>
>>> Then the compile fails says that the commands implementations of those 
>>> interfaces in package A don't match signature, because they are from two 
>>> vendor dirs.
>>>
>>> A way around this?
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[tonywalker . uk]

Ah ok. So I managed to get something to compile (ended up having to cast 
the 'const char' arguments to fopen64 to char before passing them to my 
go_fopen64 function but I still get:

fatal: morestack on g0
zsh: trace trap  LD_PRELOAD=./preload python test.py foo

I'm clearly out of my depth here but thanks for your help!

On Monday, January 15, 2018 at 3:43:40 PM UTC-5, Ian Lance Taylor wrote:
>
> On Mon, Jan 15, 2018 at 12:36 PM,   
> wrote: 
> > Thanks Ian. I assume I'd include it like this: 
> > // #include myfopen.h 
> > And also a myfopen.c with the actual function 
> > 
> > But how do I ensure the fopen64 in myfopen.h is exported to override the 
> > default? 
>
> The fopen64 function written in myfopen.c will follow ordinary C 
> rules.  If you don't make it static, it will be exported. 
>
> Ian 
>
> > On Monday, January 15, 2018 at 3:05:32 PM UTC-5, Ian Lance Taylor wrote: 
> >> 
> >> The simplist workaround is to export go_fopen64, and write a tiny 
> >> fopen64 in a different C file, with the right signature, that calls 
> >> go_fopen64. 
> >> 
> >> Ian 
> >> 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[Ian Lance Taylor]

On Mon, Jan 15, 2018 at 12:36 PM,   wrote:
> Thanks Ian. I assume I'd include it like this:
> // #include myfopen.h
> And also a myfopen.c with the actual function
>
> But how do I ensure the fopen64 in myfopen.h is exported to override the
> default?

The fopen64 function written in myfopen.c will follow ordinary C
rules.  If you don't make it static, it will be exported.

Ian

> On Monday, January 15, 2018 at 3:05:32 PM UTC-5, Ian Lance Taylor wrote:
>>
>> The simplist workaround is to export go_fopen64, and write a tiny
>> fopen64 in a different C file, with the right signature, that calls
>> go_fopen64.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[tonywalker . uk]

Thanks Ian. I assume I'd include it like this:
// #include myfopen.h
And also a myfopen.c with the actual function

But how do I ensure the fopen64 in myfopen.h is exported to override the 
default? 

On Monday, January 15, 2018 at 3:05:32 PM UTC-5, Ian Lance Taylor wrote:
>
> The simplist workaround is to export go_fopen64, and write a tiny 
> fopen64 in a different C file, with the right signature, that calls 
> go_fopen64. 
>
> Ian 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Experience Report building OCR in Go

[Tad Vizbaras]

Experience Report building OCR in Go:

http://recoink.com/goreport


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[Ian Lance Taylor]

On Mon, Jan 15, 2018 at 11:54 AM,   wrote:
>
> Thanks Ian. This did the trick and now I'm faced with the problem of
> function signatures that include 'const'. On my system, fopen is declared
> thus:
> extern FILE *fopen (char *p0, char *modes) __wur
> But fopen64 is:
> extern FILE *fopen64 (const char *__restrict __filename, const char
> *__restrict __modes) __wur
>
> So, now my override function conflicts:
>
> In file included from _cgo_export.c:3:0:
> cgo-gcc-export-header-prolog:44:14: error: conflicting types for ‘fopen64’
> In file included from ./main.go:5:0,
>  from _cgo_export.c:3:
> /usr/include/stdio.h:298:14: note: previous declaration of ‘fopen64’ was
> here
>  extern FILE *fopen64 (const char *__restrict __filename,
>   ^~~
> _cgo_export.c:37:7: error: conflicting types for ‘fopen64’
>  FILE* fopen64(char* p0, char* p1)
>^~~
>
> I'm guessing there's not much I can do about the limitation of dealing with
> 'const' declarations in C?

The simplist workaround is to export go_fopen64, and write a tiny
fopen64 in a different C file, with the right signature, that calls
go_fopen64.

Ian



> On Monday, January 15, 2018 at 2:35:19 PM UTC-5, Ian Lance Taylor wrote:
>>
>> You neglected to say which system you are running on.  If it's
>> GNU/Linux, note that fopen64 is not declared by default.  It's only
>> declared if define the C preprocessor macro _LARGEFILE64_SOURCE while
>> compiling the file.  You may need a #cgo CFLAGS: line to add a -D
>> option.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[tonywalker . uk]

Thanks Ian. This did the trick and now I'm faced with the problem of 
function signatures that include 'const'. On my system, fopen is declared 
thus:
extern FILE *fopen (char *p0, char *modes) __wur
But fopen64 is:
extern FILE *fopen64 (const char *__restrict __filename, const char 
*__restrict __modes) __wur

So, now my override function conflicts:

In file included from _cgo_export.c:3:0:
cgo-gcc-export-header-prolog:44:14: error: conflicting types for ‘fopen64’
In file included from ./main.go:5:0,
 from _cgo_export.c:3:
/usr/include/stdio.h:298:14: note: previous declaration of ‘fopen64’ was 
here
 extern FILE *fopen64 (const char *__restrict __filename,
  ^~~
_cgo_export.c:37:7: error: conflicting types for ‘fopen64’
 FILE* fopen64(char* p0, char* p1) 
   ^~~

I'm guessing there's not much I can do about the limitation of dealing with 
'const' declarations in C?

On Monday, January 15, 2018 at 2:35:19 PM UTC-5, Ian Lance Taylor wrote:
>
> You neglected to say which system you are running on.  If it's 
> GNU/Linux, note that fopen64 is not declared by default.  It's only 
> declared if define the C preprocessor macro _LARGEFILE64_SOURCE while 
> compiling the file.  You may need a #cgo CFLAGS: line to add a -D 
> option. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected behaviour with LD_PRELOAD and fopen and cgo

[Ian Lance Taylor]

On Mon, Jan 15, 2018 at 9:09 AM,   wrote:
>
> I'm doing more experiments with created shared-object files with functions
> that override syscalls and I've hit something I'd like to see if you guys
> can help with.
>
> See a cut-down version of what I'm doing here
>
> There are 2 problems..
>
> 1. If I run LD_PRELOAD=my.so ls - ls hangs completely. If however I remove
> the reference to fopen and rebuild the library - it works fine.

No idea.  You'll have to debug this to see where it is going wrong.

> 2. When I compile I get "could not determine kind of name for C.fopen64".
> Both fopen and fopen64 are defined within my stdio header file so why
> doesn't the compiler complain about fopen? If I change line 26 to fopen I
> can get round this problem but will likely hit issues with large files.

You neglected to say which system you are running on.  If it's
GNU/Linux, note that fopen64 is not declared by default.  It's only
declared if define the C preprocessor macro _LARGEFILE64_SOURCE while
compiling the file.  You may need a #cgo CFLAGS: line to add a -D
option.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected net/http context cancellation

[jesse . hallam]

Thank you -- that makes sense. I was initially thrown off by the fact that 
the connection remains open for netcat to get the full HTTP response.

For posterity, another way to get the desired behaviour is to invoke netcat 
as follows, noting that it won't exit by itself anymore:

> cat ~/tmp/payload - | nc localhost 8080


On Monday, January 15, 2018 at 12:49:05 PM UTC-5, Axel Wagner wrote:
>
> I assume netcat closes the connection once it gets an EOF from stdin. Thus 
> the server notices the client disconnecting and cancels the context, as 
> advertised.
> Try running nc without piping the input in and typing it out on the 
> console.
>
> On Mon, Jan 15, 2018 at 5:07 PM,  
> wrote:
>
>> According to https://golang.org/pkg/net/http/#Request.Context:
>>
>> For incoming server requests, the context is canceled when the client's 
>>> connection closes, the request is canceled (with HTTP/2), or when the 
>>> ServeHTTP method returns.
>>
>>
>> My expectation, then, is for the following code to block:
>>
>> package main
>>
>> import (
>> "fmt"
>> "io/ioutil"
>> "net/http"
>> )
>>
>> func main() {
>> s := {
>> Addr: ":8080",
>> Handler: http.HandlerFunc(func(w http.ResponseWriter, r *
>> http.Request) {
>> ioutil.ReadAll(r.Body)
>>
>> <-r.Context().Done()
>>
>> w.Write([]byte(fmt.Sprintf(
>> "request context done: %+v\n",
>> r.Context().Err(),
>> )))
>> }),
>> }
>> s.ListenAndServe()
>> }
>>
>>
>>
>> Instead, it always returns immediately, reporting that the context was 
>> canceled:
>>
>> > printf "POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: 
>> 4\r\n\r\nbody" | nc localhost 8080
>> HTTP/1.1 200 OK
>> Date: Mon, 15 Jan 2018 16:01:46 GMT
>> Content-Length: 39
>> Content-Type: text/plain; charset=utf-8
>>
>>
>> request context done: context canceled
>>
>>
>> Curiously, if I don't ioutil.ReadAll(r.Body), the request blocks as 
>> expected.
>>
>> Am I missing something? My hope was to be able to use the default context 
>> to detect client disconnection.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unexpected net/http context cancellation

['Axel Wagner' via golang-nuts]

I assume netcat closes the connection once it gets an EOF from stdin. Thus
the server notices the client disconnecting and cancels the context, as
advertised.
Try running nc without piping the input in and typing it out on the console.

On Mon, Jan 15, 2018 at 5:07 PM,  wrote:

> According to https://golang.org/pkg/net/http/#Request.Context:
>
> For incoming server requests, the context is canceled when the client's
>> connection closes, the request is canceled (with HTTP/2), or when the
>> ServeHTTP method returns.
>
>
> My expectation, then, is for the following code to block:
>
> package main
>
> import (
> "fmt"
> "io/ioutil"
> "net/http"
> )
>
> func main() {
> s := {
> Addr: ":8080",
> Handler: http.HandlerFunc(func(w http.ResponseWriter, r *
> http.Request) {
> ioutil.ReadAll(r.Body)
>
> <-r.Context().Done()
>
> w.Write([]byte(fmt.Sprintf(
> "request context done: %+v\n",
> r.Context().Err(),
> )))
> }),
> }
> s.ListenAndServe()
> }
>
>
>
> Instead, it always returns immediately, reporting that the context was
> canceled:
>
> > printf "POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length:
> 4\r\n\r\nbody" | nc localhost 8080
> HTTP/1.1 200 OK
> Date: Mon, 15 Jan 2018 16:01:46 GMT
> Content-Length: 39
> Content-Type: text/plain; charset=utf-8
>
>
> request context done: context canceled
>
>
> Curiously, if I don't ioutil.ReadAll(r.Body), the request blocks as
> expected.
>
> Am I missing something? My hope was to be able to use the default context
> to detect client disconnection.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] how about golang channel memory usage?

[Jan Mercl]

On Mon, Jan 15, 2018 at 6:04 PM Lynn H  wrote:

> so what the rules of channel size and memory usage?

Check [0] if the formulas are correct.

  [0]:
https://docs.google.com/spreadsheets/d/1BuAnHmBKWfWD9JXXi4_k9ck-rkJyDuQgCQ05s4ytJI8/edit?usp=sharing



-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Directory structure question

[Kim Nielsen]

Hi,

This is not to start a flamewar but I have been using go for a while now 
and then I stumpled upon a question about the way one should structure a 
workspace that seems to be different from several articles, the 
documentation and they way people structure their github repositories.

First question:

GOPATH is normally ~/go unless otherwise. So by using go get like go get 
github.com/golang/example/hello it adds the files in 
~/go/src/github.com/golang/examples/hello. But if you are using a CI system 
that needs to check the entire system out with access from the internet its 
seems to be easier to structure the project like

src/
   /hello/
 /hello.go
bin/
pkg/
internal/
vendor/
  /github.com/golang/example/stringutil/
Makefile

and then make the Makefile set GOPATH=$(shell pwd). But is this best 
practice when the CI server does not have access to the internet?

Second question:

In the documentation on golang.org there is an example of using the pkg 
where a .a file is kept after building. But several others who have been 
long term members of the Go community have best practice where they place 
source files under pkg. It seems that there is a disagreement on what 
should be put in there. The only thing I can find that everyone agrees on 
is that its for packages. But should one place local non exported packages 
there or is the internal folder better for that ?

All of the above solutions work but I'd really like someones oppinion on 
what is best regarding future use of go tools.

Best regards
Kim

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] how about golang channel memory usage?

[Lynn H]

I user golang to write tcp  server recently, I'm puzzled by  memoy usage 
with channel

so I write some code for test
   
import (
"time"
)

func main() {
memsize := 256
for i := 0; i < 10; i++ {
go func() {
m := make(chan []byte, memsize)
time.Sleep(30 * time.Second)
<-m
}()
}
time.Sleep(30 * time.Second)
}
 I user  RES value of 'top' command to get program memory use 
  memsize = 0,  program use 258M (memory just use to  allocate for 10 
routines)
  memsize = 256,program use 306M, 10 channel use 48M
  memsize = 512, program use 328M,10 channel use 70M
  memsize = 1024,program use 379M, 10 channel use 121M
  memsize = 2048,program use 455M, 10 channel use 197M

so what the rules of channel size and  memory usage?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Unexpected net/http context cancellation

[jesse . hallam]

According to https://golang.org/pkg/net/http/#Request.Context:

For incoming server requests, the context is canceled when the client's 
> connection closes, the request is canceled (with HTTP/2), or when the 
> ServeHTTP method returns.


My expectation, then, is for the following code to block:

package main

import (
"fmt"
"io/ioutil"
"net/http"
)

func main() {
s := {
Addr: ":8080",
Handler: http.HandlerFunc(func(w http.ResponseWriter, r *
http.Request) {
ioutil.ReadAll(r.Body)

<-r.Context().Done()

w.Write([]byte(fmt.Sprintf(
"request context done: %+v\n",
r.Context().Err(),
)))
}),
}
s.ListenAndServe()
}



Instead, it always returns immediately, reporting that the context was 
canceled:

> printf "POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: 
4\r\n\r\nbody" | nc localhost 8080
HTTP/1.1 200 OK
Date: Mon, 15 Jan 2018 16:01:46 GMT
Content-Length: 39
Content-Type: text/plain; charset=utf-8


request context done: context canceled


Curiously, if I don't ioutil.ReadAll(r.Body), the request blocks as 
expected.

Am I missing something? My hope was to be able to use the default context 
to detect client disconnection.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: How to implement Segment Tree in key/value datastore to use Google S2 geometry?

[Jesper Louis Andersen]

My intuition is that this screams S2 at the top of its lungs, provided that
you can hook into the structure.

It is simply storing a radix tree (PATRICIA) of S2 cells and attaching
information about box bounding to all of these. This yields a compressed
trie-like structure that you can probably maintain in memory even for
fairly large sets of bounding boxes. The path compression means that you
need considerably fewer than the 30-cell lookup depth that would normally
happen. If they haven't this built-in I don't know why, as it seems rather
obvious to me, givien that you know the internal structure of S2[0].

The alternative ideas are just quad-tree'ing the whole world, but then you
are getting close to an R-tree solution anyway. Another solution is to just
brute-force everything on two hash-table maps. Uber did this in a blog
post, but I wouldn't recommend that solution since it won't really scale.
Finally, you can just stuff everything into PostGIS in Postgres, which
contains many of these algorithms already. If you need to link to other
relational data, you shouldn't rule out such a solution I think.

[0] If you've never looked into this, do it. It is awfully brilliantly
designed. Hilbert curves and all.

On Sun, Jan 14, 2018 at 7:50 PM Constantine Vassilev 
wrote:

> Thanks for the suggestion and the link.
>
> The question is how it will be searched. Also I liked the Google S2's
> Golang implementation.
>
> From the documentation if the link I see:
> --
> Queries
>
> Bounding-box and k-nearest-neighbors queries are supported.
>
> Bounding-box queries require a search *Rect. It returns all objects that
> touch the search rectangle.
>
> --
>
> What I need is having location's lat/lon to find the polygon around it.
>
> Also how the index would be created and stored as key/value.
>
> On Saturday, January 13, 2018 at 11:21:48 PM UTC-8, Tamás Gulácsi wrote:
>>
>>
>> 2018. január 14., vasárnap 5:48:11 UTC+1 időpontban Constantine Vassilev
>> a következőt írta:
>>>
>>> I need to implement the following:
>>>
>>> 1) I have a database of ~100 mil records.
>>>
>>> Each record is of type key/value.
>>>
>>> The key is an internal ID of type: "123456789".
>>> The value is a kml with geographic boundary in form of lat/lon
>>> coordinates:
>>>
>>>  
>>>  http://www.opengis.net/kml/2.2\;>
>>>
>>>
>>>  
>>>  
>>>
>>>
>>> -117.587174583104,33.6437620515745
>>>
>>> -117.587083176312,33.6437891755388
>>>
>>> -117.587016626677,33.6438098355405
>>>
>>> -117.586923617992,33.6435949397585
>>>
>>> -117.587051757569,33.643539681552
>>>
>>> -117.587079610599,33.6435348310862
>>>
>>> -117.587174583104,33.6437620515745
>>>
>>>  
>>>   
>>> 
>>> 
>>>   
>>>
>>>
>>> 2) Input values are of type lat/lon like -117.1,33.
>>>
>>> The usage is having an input lat/lon to find the boundary in which it is
>>> located and return the key (the internal ID).
>>>
>>> The closest solution I found is using Google S2 geometry library:
>>> https://github.com/golang/geo
>>> and this library:
>>> https://github.com/akhenakh/regionagogo
>>>
>>> This implementation is using an in memory index with Segment Tree
>>>  datastructure:
>>> github.com/Workiva/go-datastructures/augmentedtree
>>> and BoltDB.
>>>
>>>
>> Why not R-tree (https://github.com/dhconnelly/rtreego) ?
>> You want poligon bounded inclusion, don't you?
>> With segmented tree, you'll need to implement some other data structure
>> that holds the slices (say, you slice the globe vertically and each slice
>> holds a segment tree of the vertically sliced poligons),
>> I think
>>
>>
>> I need to create persistent  key/value database on the disk (it would be
>>> big like ~100GB) using
>>> RocksDB key/value store with key Segment Tree
>>>  datastructure.
>>>
>>> It should get CellID from lat/lon and find the S2 interval where it
>>> resides.
>>>
>>> How is the best way to implement it?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Measuring low latency operations

[Jesper Louis Andersen]

General rules:

* If possible, use the benchmark system inside the "testing" package. It
will eliminate some common mistakes in benchmarking, but keep your head
straight while doing it, so you don't accidentally measure setup time, or
something else which is outside the thing you are trying to test.

* A fast benchmark is sensitive to variance on the machine. The "testing"
packages smooths it out by running the test more than once, but another
trick is to compute outlier variance through bootstrapping methods and
figure out if the result is likely to be skewed by outliers.

* Likely, variance can occur from multiple sources:
  - Servers which are virtualized compete with resources from neighboring
systems.
  - Desktop computers can have your benchmark compete with e.g., your music
player or browsing while tests are being run.

* When you report numbers, beware of the average. At least report the
median, and if it differs from the average, you should be worried since the
data set is unlikely to be normally distributed.

* Prefer visual methods: Compute a kernel density plot of the data to see
how it distributes. You want to look out for odd shapes.

* If your data sets have the same variance, are following a normal
distribution and are independent, you can compare them with e.g.,
student's-t. FreeBSD has the ministat(1) tool by Poul-Henning Kamp for
this, and I know it is easily ported. But a bit of R or Python also does
the trick.



On Mon, Jan 15, 2018 at 3:07 AM  wrote:

> All:
>
> I am testing an application that has a latency of the order of few to
> several microseconds.  I wrote a benchmark and measured the performance.  I
> also computed the elapsed time using time.Now() and time.Since().  (I will
> change this to time,Nanoseconds and try shortly.
>
> I found the computed latencies had a large variation.
>
> Set size: 22024
>
> Min: 20, Max: 936 Average: 32 SD: 43 all in microseconds
>
>
>  It's skewed towards the lower end of the spectrum.  go test itself
> reported 56 microseconds as the response time.
>
>1. Is this variation due to garbage collection?
>1. If so, can the variation be somehow reduced by tuning the garbage
>   collector?
>
>
> I will appreciate any references or thoughts.
>
> With regards,
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.