[go-nuts] caching of cgo compilation
Under what conditions does a cgo package get recompiled? I am using go-sqlite3 and it appears that the C compiler/linker is run every time I build my project, even though I am not changing any of the C sources. I would have expected it to be run once and then the results re-used, but perhaps there are some special rules for cgo recompilation? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] caching of cgo compilation
Hmm but I am also not changing the Go source of the package in question, yet the C sources seem to get recompiled. Just to make sure we're on the same page, I have two packages: mypkg and go-sqlite3, where the former is pure Go and the depends on the latter, which contains both C and Go code. Whenever I make changes to Go sources in mypkg I can see that the cgo compiler is being run on go-sqlite3. On Wed, Jun 15, 2016 at 3:23 PM Ian Lance Taylor wrote: > On Wed, Jun 15, 2016 at 2:54 PM, Alex Flint wrote: > > Under what conditions does a cgo package get recompiled? I am using > > go-sqlite3 and it appears that the C compiler/linker is run every time I > > build my project, even though I am not changing any of the C sources. I > > would have expected it to be run once and then the results re-used, but > > perhaps there are some special rules for cgo recompilation? > > Currently the go tool either compiles an entire package or doesn't > compile it at all. If it is a cgo package with .c files, it compiles > those .c files every time it thinks any part of the package needs to > be recompiled--e.g., if you changed the Go code. > > I thought there was an issue open about that, but I couldn't find it. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] semantics of byteCount in gob encoding
The gob docs state that a gob stream consists of
(byteCount (-type id, encoding of a wireType)* (type id, encoding of a
value))*
I was expecting byteCount to be the number of bytes remaining in the entire
packet, but that does not seem to be the case. For example when encoding a
single instance of "struct Number { X int }", the gob stream is
000 1a ff 81 03 01 01 06 4e 75 6d 62 65 72 01 ff 82
010 00 01 01 01 01 58 01 04 00 00 00 05 ff 82 01 06
020 00
The entire stream is 33 bytes, so I was expecting the first byte to be
0x20, but in fact it is 0x1a. Is this count exclusive of the "(-type id,
encoding of a wireType)*" segment?
Alex
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] semantics of byteCount in gob encoding
Oh I expected that since I only called encoder.Encode once, there would
only be on repetition of the outermost segment. I was expecting the
wireType to be within the inner "(-type id, encoding of a wireType)*"
segment. Otherwise shouldn't the spec be
((byteCount, -type id, encoding of a wireType)* (byteCount, type id,
encoding of a value))*
?
On Tue, Jun 21, 2016 at 8:47 AM Rob Pike wrote:
> It's as advertised, 1 count byte (1a=26) followed by 26 bytes, followed by
> one count byte (5), followed by 5 bytes. Note the final '*' in that grammar.
> -rob
>
>
> On Tue, Jun 21, 2016 at 8:20 AM, Alex Flint wrote:
>
>> The gob docs state that a gob stream consists of
>>
>> (byteCount (-type id, encoding of a wireType)* (type id, encoding of
>> a value))*
>>
>> I was expecting byteCount to be the number of bytes remaining in the
>> entire packet, but that does not seem to be the case. For example when
>> encoding a single instance of "struct Number { X int }", the gob stream is
>>
>> 000 1a ff 81 03 01 01 06 4e 75 6d 62 65 72 01 ff 82
>> 010 00 01 01 01 01 58 01 04 00 00 00 05 ff 82 01 06
>> 020 00
>>
>> The entire stream is 33 bytes, so I was expecting the first byte to be
>> 0x20, but in fact it is 0x1a. Is this count exclusive of the "(-type id,
>> encoding of a wireType)*" segment?
>>
>> Alex
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] semantics of byteCount in gob encoding
Sorry to keep bothering you folks with this but does anybody have an answer
to this?
On Tue, Jun 21, 2016 at 9:26 AM Alex Flint wrote:
> Oh I expected that since I only called encoder.Encode once, there would
> only be on repetition of the outermost segment. I was expecting the
> wireType to be within the inner "(-type id, encoding of a wireType)*"
> segment. Otherwise shouldn't the spec be
>
> ((byteCount, -type id, encoding of a wireType)* (byteCount, type id,
> encoding of a value))*
>
> ?
>
> On Tue, Jun 21, 2016 at 8:47 AM Rob Pike wrote:
>
>> It's as advertised, 1 count byte (1a=26) followed by 26 bytes, followed
>> by one count byte (5), followed by 5 bytes. Note the final '*' in that
>> grammar.
>> -rob
>>
>>
>> On Tue, Jun 21, 2016 at 8:20 AM, Alex Flint wrote:
>>
>>> The gob docs state that a gob stream consists of
>>>
>>> (byteCount (-type id, encoding of a wireType)* (type id, encoding of
>>> a value))*
>>>
>>> I was expecting byteCount to be the number of bytes remaining in the
>>> entire packet, but that does not seem to be the case. For example when
>>> encoding a single instance of "struct Number { X int }", the gob stream is
>>>
>>> 000 1a ff 81 03 01 01 06 4e 75 6d 62 65 72 01 ff 82
>>> 010 00 01 01 01 01 58 01 04 00 00 00 05 ff 82 01 06
>>> 020 00
>>>
>>> The entire stream is 33 bytes, so I was expecting the first byte to be
>>> 0x20, but in fact it is 0x1a. Is this count exclusive of the "(-type
>>> id, encoding of a wireType)*" segment?
>>>
>>> Alex
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] getting the receiver from a bound method
Is it possible to use reflection to retrieve the value of x from reflect.ValueOf(x.SomeMethod) where x is an instance of some struct type? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] gob-encoding pointer to zero
Is it intended that gob-encoding a pointer to an integer with value equal to zero decodes as a nil pointer? I was expecting to get back a non-nil pointer to an integer with value zero. https://play.golang.org/p/UUN0UFEeIN I understand that gob omits zero values but my understanding was that a non-nil pointer was not a zero value. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] compressing long list of short strings
I have long list of short strings that I want to compress, but I want to be able to decompress an arbitrary string in the list at any time without decompressing the entire list. I know the list ahead of time and it doesn't matter how much preprocessing time is involved. It is also fine if there is some significant O(1) memory overhead at runtime. Any suggestions? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] compressing long list of short strings
There are around 2M strings, and their total size is ~6 GB, so an average of 3k each. I actually looked briefly at Go's compress/flate to see whether something like what you're describing is possible without writing my own compressor but I couldn't see any obvious way to get at the underlying compressor state. Or perhaps I'm looking in the wrong package - any pointers would be appreciated. On Wed, Aug 10, 2016 at 3:42 PM Ian Lance Taylor wrote: > On Wed, Aug 10, 2016 at 3:27 PM, Alex Flint wrote: > > > > I have long list of short strings that I want to compress, but I want to > be > > able to decompress an arbitrary string in the list at any time without > > decompressing the entire list. > > > > I know the list ahead of time and it doesn't matter how much > preprocessing > > time is involved. It is also fine if there is some significant O(1) > memory > > overhead at runtime. > > > > Any suggestions? > > You say the strings are "short": how short? How many strings are > there? How much total data in the uncompressed strings? > > What is your target for the total amount of memory used by the > compressed strings plus any data required to decompress them? > > One approach that comes to mind is building an optimized Huffman table > for the full set of strings, and compressing each one separately using > that table. Then each string is represented by a bit offset into the > resulting bitstream, and each can be decompressed separately. But you > would need storage at run time not only for the bitstream, but also > for the Huffman table. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] cgo and OSX "main thread"
I am using cgo to wrap a library that creates an OSX UI, which requires things to be run on the "main thread". Sometimes my cgo calls end up on the main thread and sometimes they do not (I check this with [NSThread isMainThread]). I understand that cgo threads are guaranteed to run on an OS-allocated stack, but what does this mean w.r.t. the OSX "main thread"? Has anyone else faced this problem before? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] make a struct pointer _not_ implement an interface
Is it possible to have a struct that implements an interface, but have pointers to the struct not implement the interface? The reason is that I want to find all the places in our codebase that attempt to use a pointer to a certain struct as a certain interface. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] interpreting in a traceback
I am looking for help understanding the following lines in a Go traceback: 6 File "specialized.go" line 163 in pythontype.ExplicitFunc.Call 7 File "" line 510 in pythontype.(*ExplicitFunc).Call 8 File "propagate.go" line 622 in pythonstatic.(*propagator).evaluateCallExpr I am trying to understand what line 7 means in the following context: evaluateCallExpr takes a parameter x of type pythontype.Value, which is an interface, and invokes x.Call(), which is one of the methods on that interface. One implementation of that interface is pythontype.ExplicitFunc. Am I right in understanding from that traceback that the dynamic type of the pythontype.Value was *pythontype.ExplicitFunc, not pythontype.ExplicitFunc? Thanks in advance! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] interpreting in a traceback
Thanks Ian. Just to confirm: it's not possible that the dynamic type of x as seen from Go code would be ExplicitFunc, but the interface stores a *ExplicitFunc internally within the itab, and so the compiler generates (*ExplicitFunc).Call? On Mon, Sep 19, 2016 at 3:46 PM Ian Lance Taylor wrote: > On Mon, Sep 19, 2016 at 3:21 PM, Alex Flint wrote: > > I am looking for help understanding the following lines in a Go > traceback: > > > > 6 File "specialized.go" line 163 in pythontype.ExplicitFunc.Call > > 7 File "" line 510 in pythontype.(*ExplicitFunc).Call > > 8 File "propagate.go" line 622 in > > pythonstatic.(*propagator).evaluateCallExpr > > > > I am trying to understand what line 7 means in the following context: > > evaluateCallExpr takes a parameter x of type pythontype.Value, which is > an > > interface, and invokes x.Call(), which is one of the methods on that > > interface. One implementation of that interface is > pythontype.ExplicitFunc. > > Am I right in understanding from that traceback that the dynamic type of > the > > pythontype.Value was *pythontype.ExplicitFunc, not > pythontype.ExplicitFunc? > > Yes. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] package containing standard CA roots?
Does anyone know of a golang package that embeds (go-bindata or similar) a reasonable standard set of CA roots? Ideally such a package would provide a ready-to-use http.Client. For context, I'm building minimal docker images containing go binaries that need to make https connections to some third party APIs. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] package containing standard CA roots?
I'm working with busybox, which does not ship with CA roots. On Sat, Dec 17, 2016 at 12:26 AM Konstantin Khomoutov < flatw...@users.sourceforge.net> wrote: > On Thu, 15 Dec 2016 16:35:09 +0000 > Alex Flint wrote: > > > Does anyone know of a golang package that embeds (go-bindata or > > similar) a reasonable standard set of CA roots? Ideally such a > > package would provide a ready-to-use http.Client. > > > > For context, I'm building minimal docker images containing go > > binaries that need to make https connections to some third party APIs. > > In such context, why would you need that? Every sensible > GNU/Linux-based OS ships a package containing such list of CA > certificates, and Go built for GOOS=linux knows how to find those certs > in a set of standard places. > > Sure, one problem with this is that the list is opinionated; on the > other hand, the list of your imaginary package would be opinionated as > well. On the other hand, whatever list is shipped with your base OS > gets security updates and also updates which merely bring the list > up-to-date (just like the time-zone information package(s)). > > So I'd just rely on the underlying OS. > In Debian and it's derivatives it's named "ca-certificates". > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] extracting transport failures in httputil.ReverseProxy
Recently while using http.ReverseProxy I wanted to take a specific action if there was a TCP connection failure or other transport-level failure in ReverseProxy.ServerHTTP. However, ReverseProxy.ServeHTTP simply sets a BadGateway status code in this situation, which makes it impossible to differentiate between an actual transport failure and a BadGateway response being returned from the upstream server to which I am proxying. For context, the action I wanted to take was to restart the backend service return an HTTP redirect to another endpoint. One attempt I made was to write my own Transport wrapper that stored the error, but this didn't work because there was no way to stop the ReverseProxy from still rendering the BadGateway response. After looking at the code, it seems that one could add func (ReverseProxy) TryServeHTTP(w, r) error which would return an error on transport failure, but not on any valid HTTP response from the upstream server. ServerHTTP would then call TryServeHTTP and set a BadGateway if err != nil. I realize there is a high bar for standard library API changes but I just thought I'd throw this out there... Alex -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] efficiency of string <-> []byte conversion
As of go1.8, do conversions between strings and byte slices always generate a copy? I understand the immutability guarantees in the language spec, but I'm interested in this from an efficiency standpoint. I can imagine a compiler that analyzes whether a byte slice created from such a conversion is ever modified and foregoing the copy in some cases, while still adhering to the immutability guarantees. I have searched this forum for a past thread on this topic but have come up empty. Feel free to point me to the relevant discussion. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: efficiency of string <-> []byte conversion
Thanks Val. I actually did a search before posting this and turned up many of those same links, but it seems like they're all just reasoning from the immutability of strings, which I don't think is quite sufficient to answer the question in all cases. Anyway, thanks for your help and I think for now I'll work on the assumption that string/byteslice conversions do always copy. On Mon, Feb 13, 2017 at 2:22 PM Val wrote: > Hello Alex, > good point. I remember this has indeed come up in discussions in this > forum. > > Searching "byte slice string copy" gave (among others) [1] [2] [3] [4] [5] > [6] [7]. All of these look related to the suggested optimization "don't > allocate© for a []byte to string, or string to []byte, conversion, > under certain circumstances". > Cheers > Val > > [1] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/m46jaS6bgZg/R5hSOPFz5m0J > [2] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/CwoKDOhv6vw/sf35VZOZDQAJ > [3] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/ENgbUzYvCuU/gWa8V8wGAwAJ > [4] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/ajXzEM6lqJI/vRsqXxizBAAJ > [5] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/f2cX-BIJsqY/bzK77xpU8PMJ > [6] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/IfILC-wllaI/U8v88T3aBgAJ > [7] > https://groups.google.com/forum/#!searchin/golang-nuts/byte$20slice$20string$20copy|sort:relevance/golang-nuts/fXmG83gZOJg/eixtKSQiNysJ > > > On Monday, February 13, 2017 at 9:31:06 PM UTC+1, Alex Flint wrote: > > As of go1.8, do conversions between strings and byte slices always > generate a copy? > > I understand the immutability guarantees in the language spec, but I'm > interested in this from an efficiency standpoint. I can imagine a compiler > that analyzes whether a byte slice created from such a conversion is ever > modified and foregoing the copy in some cases, while still adhering to the > immutability guarantees. > > I have searched this forum for a past thread on this topic but have come > up empty. Feel free to point me to the relevant discussion. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "golang-nuts" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/golang-nuts/C6BBNTRSEwg/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] contexts for busy computations
Suppose I have a long-running computation that I wish to be cancelable via a context. By this I mean something that's just churning away on some computation, not blocking on a syscall or I/O operation or anything like that. Should I just poll ctx.Err() periodically in the inner loop of my computation? Also, can anyone give any insight into approximately how costly context.Err() is for the contexts constructed in the context package -- I don't want my computation to be dominated by checking for cancellation! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
