Re: [go-nuts] Context cancellation

[Gustavo Niemeyer]

Glad it's being useful, and indeed, that API looks familiar. Nice.

On Nov 3, 2016 4:33 PM, "Chris Hines"  wrote:

> FWIW, I'm a big fan of gopkg.in/tomb.v2. For similar functionality using
> contexts consider golang.org/x/sync/errgroup.
>
> On Wednesday, November 2, 2016 at 7:55:59 PM UTC-4, Gustavo Niemeyer wrote:
>>
>> You said the function would return whether it got cancelled or not, so I
>> assumed something like err := foo.CallAPI(ctx), which would be awkward to
>> return without actually canceling.
>>
>> Yes, if the function is documented to return with background activity,
>> it's of course fine. That said, It's extremely helpful to be able to
>> reliably cancel such background activity and wait until it's done, for a
>> number of reasons: side effect control, testing, polite shutdowns, etc.
>>
>> That's mainly where gopkg.in/tomb.v2 comes from.
>>
>> On Nov 3, 2016 1:43 AM, "Axel Wagner"  wrote:
>>
>> In a concurrent world, assuming that a call to cancel means that thing
>> actually got cancelled is dubious at best. For example, you could call
>> cancelFunc, then immediately enter a GC pause, the other goroutine finishes
>> their work in a orderly fashion, you unpause and close the underlying
>> channel.
>> Very normal behavior, perfectly valid code, unpreventable situation. But
>> still, calling cancelFunc doesn't mean it actually had any real effect.
>> This is what I mean by "successful cancellation" (or not).
>>
>> I don't believe we actually disagree. Though there might be some phrasing
>> issue. Which is now adding more noise and confusion (sorry. I'll shut up
>> now).
>>
>> On Thu, Nov 3, 2016 at 12:31 AM, Gustavo Niemeyer 
>> wrote:
>>
>>>
>>>
>>> On Thu, Nov 3, 2016 at 1:27 AM, Axel Wagner 
>>> wrote:
>>>
>>>> That is actually a great point I haven't thought about and the crux of
>>>> the matter (sorry for repeating it, but I think it's worth making very
>>>> explicit):
>>>>
>>>> While cancelFunc can only be called from the goroutine that created the
>>>> context, Err can be called downstack. Meaning, if I call cancelFunc, a call
>>>> *might or might not* return Cancelled as an error. It might not actually
>>>> implement cancellation or the call might have failed or succeeded
>>>> concurrently with a different (non-)error.
>>>>
>>>> The ctx.Err() return thus allows a "child-function" to signal back
>>>> whether a cancellation was *successful*; the creator of the context only
>>>> know that it has been *tried*.
>>>>
>>>
>>> No, that's really not its intent. Cancel means *STOP IT!*, and any
>>> goroutines down the chain are supposed to block until they're really done,
>>> returning ASAP.
>>>
>>> It's a nasty bug to disrespect that, because the call site will assume
>>> that the background noise stopped once the function returned, and act
>>> accordingly thereafter.
>>>
>>>
>>> gustavo @ http://niemeyer.net
>>>
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Context cancellation

[Gustavo Niemeyer]

You said the function would return whether it got cancelled or not, so I
assumed something like err := foo.CallAPI(ctx), which would be awkward to
return without actually canceling.

Yes, if the function is documented to return with background activity, it's
of course fine. That said, It's extremely helpful to be able to reliably
cancel such background activity and wait until it's done, for a number of
reasons: side effect control, testing, polite shutdowns, etc.

That's mainly where gopkg.in/tomb.v2 comes from.

On Nov 3, 2016 1:43 AM, "Axel Wagner"  wrote:

In a concurrent world, assuming that a call to cancel means that thing
actually got cancelled is dubious at best. For example, you could call
cancelFunc, then immediately enter a GC pause, the other goroutine finishes
their work in a orderly fashion, you unpause and close the underlying
channel.
Very normal behavior, perfectly valid code, unpreventable situation. But
still, calling cancelFunc doesn't mean it actually had any real effect.
This is what I mean by "successful cancellation" (or not).

I don't believe we actually disagree. Though there might be some phrasing
issue. Which is now adding more noise and confusion (sorry. I'll shut up
now).

On Thu, Nov 3, 2016 at 12:31 AM, Gustavo Niemeyer 
wrote:

>
>
> On Thu, Nov 3, 2016 at 1:27 AM, Axel Wagner  > wrote:
>
>> That is actually a great point I haven't thought about and the crux of
>> the matter (sorry for repeating it, but I think it's worth making very
>> explicit):
>>
>> While cancelFunc can only be called from the goroutine that created the
>> context, Err can be called downstack. Meaning, if I call cancelFunc, a call
>> *might or might not* return Cancelled as an error. It might not actually
>> implement cancellation or the call might have failed or succeeded
>> concurrently with a different (non-)error.
>>
>> The ctx.Err() return thus allows a "child-function" to signal back
>> whether a cancellation was *successful*; the creator of the context only
>> know that it has been *tried*.
>>
>
> No, that's really not its intent. Cancel means *STOP IT!*, and any
> goroutines down the chain are supposed to block until they're really done,
> returning ASAP.
>
> It's a nasty bug to disrespect that, because the call site will assume
> that the background noise stopped once the function returned, and act
> accordingly thereafter.
>
>
> gustavo @ http://niemeyer.net
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Context cancellation

[Gustavo Niemeyer]

On Thu, Nov 3, 2016 at 1:27 AM, Axel Wagner 
wrote:

> That is actually a great point I haven't thought about and the crux of the
> matter (sorry for repeating it, but I think it's worth making very
> explicit):
>
> While cancelFunc can only be called from the goroutine that created the
> context, Err can be called downstack. Meaning, if I call cancelFunc, a call
> *might or might not* return Cancelled as an error. It might not actually
> implement cancellation or the call might have failed or succeeded
> concurrently with a different (non-)error.
>
> The ctx.Err() return thus allows a "child-function" to signal back whether
> a cancellation was *successful*; the creator of the context only know that
> it has been *tried*.
>

No, that's really not its intent. Cancel means *STOP IT!*, and any
goroutines down the chain are supposed to block until they're really done,
returning ASAP.

It's a nasty bug to disrespect that, because the call site will assume that
the background noise stopped once the function returned, and act
accordingly thereafter.


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Context cancellation

[Gustavo Niemeyer]

On Wed, Nov 2, 2016 at 9:11 PM, Ian Davis  wrote:

> On Wed, Nov 2, 2016, at 10:35 PM, Gustavo Niemeyer wrote:
>
> Hello there,
>
> On Wed, Nov 2, 2016 at 11:09 AM, Ian Davis  wrote:
>
>
>
> On Wed, Nov 2, 2016, at 12:56 PM, 'Axel Wagner' via golang-nuts wrote:
>
> AIUI: A child or grandchild function is not supposed to signal that. They
> can return an error and let the parent cancel, or they can create their own
> child context WithCancel and cancel that. Context doesn't replace
> exceptions.
>
>
>
> OK I see that. I think I don't understand the purpose of the Err method if
> the function that creates the context is the one that cancels it or closes
> the Done channel. It seems that the only purpose would be to detect a
> Timeout.
>
>
> Err allows you to do this:
>
> select {
> case <-ctx.Done():
> return ctx.Err()
> ...
> }
>
>
> In this situation, other than a timeout, what would be closing the Done
> channel? The goroutine that created the context already knows the error
> since it's the only one that is supposed to call cancel.
>

The goroutine that created the context closes the channel. The error
returned informs the call site why the task was interrupted, and that call
site will often not be the calling goroutine. Without it you'd have to
return fmt.Errorf("interrupted") or something, which is both more boring
and less precise than simply calling ctx.Err().



> You should consider context a way to implement dynamic scoping
> <https://en.wikipedia.org/wiki/Scope_(computer_science)#Dynamic_scoping>.
> A context is a thing valid for the current stack frame and downwards and
> should never affect the stack upwards. Which is also why there's the "put
> values in there, not pointers" rule; if you have a pointer, it might modify
> an upstack context.
>
> The advantage of that (and context in general) is, that you don't need to
> worry too much about synchronization and everything; as a context is
> fundamentally immutable (-ish), it's fine to pass it to separate
> goroutines, because they can only do stuff with it downstack.
>
>
> Yes, this is what I am trying to do. I was wondering whether the returned
> cancelFunc is something that could be passed downstream to signal parents
> that work should be aborted.
>
>
> No, the purpose of Context is the opposite: fired tasks are free to do
> whatever they please until it's time to stop, because the boss says so.
> Once that happens, error propagation happens as usual to tell what went
> wrong.
>
> The http://gopkg.in/tomb.v2 package works closer to the model you
> describe.
>
>
> Thanks. I've used tomb before and I've played with the context/tomb
> integration. I guess I need both.
>

Yeah, tomb and context have some overlap, but they cover different needs.
Pretty much every use I made of tomb was on an internal implementation
detail to organize the required concurrency inside a service, while context
is very nice across API boundaries.


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Context cancellation

[Gustavo Niemeyer]

Hello there,

On Wed, Nov 2, 2016 at 11:09 AM, Ian Davis  wrote:

> On Wed, Nov 2, 2016, at 12:56 PM, 'Axel Wagner' via golang-nuts wrote:
>
> AIUI: A child or grandchild function is not supposed to signal that. They
> can return an error and let the parent cancel, or they can create their own
> child context WithCancel and cancel that. Context doesn't replace
> exceptions.
>
>
> OK I see that. I think I don't understand the purpose of the Err method if
> the function that creates the context is the one that cancels it or closes
> the Done channel. It seems that the only purpose would be to detect a
> Timeout.
>

Err allows you to do this:

select {
case <-ctx.Done():
return ctx.Err()
...
}


> You should consider context a way to implement dynamic scoping
> .
> A context is a thing valid for the current stack frame and downwards and
> should never affect the stack upwards. Which is also why there's the "put
> values in there, not pointers" rule; if you have a pointer, it might modify
> an upstack context.
>
> The advantage of that (and context in general) is, that you don't need to
> worry too much about synchronization and everything; as a context is
> fundamentally immutable (-ish), it's fine to pass it to separate
> goroutines, because they can only do stuff with it downstack.
>
> Yes, this is what I am trying to do. I was wondering whether the returned
> cancelFunc is something that could be passed downstream to signal parents
> that work should be aborted.
>

No, the purpose of Context is the opposite: fired tasks are free to do
whatever they please until it's time to stop, because the boss says so.
Once that happens, error propagation happens as usual to tell what went
wrong.

The http://gopkg.in/tomb.v2 package works closer to the model you describe.


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] gopkg.in updated: snap, Let's Encrypt, etc

[Gustavo Niemeyer]

There are some conversations in the wild related to that, but I have no
proper information or opinion about it.

The limitations I described are purely technical. The git binary in
distributions as recent as 2014 linked with a gnutls library that was
unable to take the available ciphers. It was working fine on more recent
distributions.

On Nov 2, 2016 8:03 PM, "Pietro Gagliardi"  wrote:

>
> On Nov 2, 2016, at 1:09 PM, Gustavo Niemeyer  wrote:
>
> The problem was that git from very recent distributions linked against
> gnutls and did not support elliptic curve certificates.
>
> Did something happen that made elliptic curve certificates undesirable?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: gopkg.in updated: snap, Let's Encrypt, etc

[Gustavo Niemeyer]

Okay, it's back up and working properly with older git clients.

The problem was that git from very recent distributions linked against
gnutls and did not support elliptic curve certificates.

Hacking the library to ask Let's Encrypt for RSA certificates instead fixed
the problem.

On Wed, Nov 2, 2016 at 9:10 AM, Gustavo Niemeyer 
wrote:

> I've reverted it temporarily as I'm observing a large number of these
> messages, certainly from old git clients:
>
> tls: no cipher suite supported by both client and server
>
> Will investigate and redeploy soon.
>
> On Wed, Nov 2, 2016 at 8:32 AM, Gustavo Niemeyer 
> wrote:
>
>> Hello all,
>>
>> After complaints about the previous StartCom TLS certificate, it was
>> moved over to Let's Encrypt's dynamic generation of certificates using the
>> autocert package [1].
>>
>> It's also been updated to be deployed as a snap [2], which means it's
>> better confined inside its system and a bit easier for me to maintain and
>> replicate.
>>
>> Finally, I also took this chance to move over the primary to a different
>> provider and datacenter. The deployment continues to have automatic
>> failover across datacenters and cities.
>>
>>
>> Please do let me know if you observe any hiccups.
>>
>>
>> [1] https://golang.org/x/crypto/acme/autocert
>> [2] http://snapcraft.io
>>
>>
>> gustavo @ http://niemeyer.net
>>
>
>
>
> --
>
> gustavo @ http://niemeyer.net
>



-- 

gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: gopkg.in updated: snap, Let's Encrypt, etc

[Gustavo Niemeyer]

I've reverted it temporarily as I'm observing a large number of these
messages, certainly from old git clients:

tls: no cipher suite supported by both client and server

Will investigate and redeploy soon.

On Wed, Nov 2, 2016 at 8:32 AM, Gustavo Niemeyer 
wrote:

> Hello all,
>
> After complaints about the previous StartCom TLS certificate, it was moved
> over to Let's Encrypt's dynamic generation of certificates using the
> autocert package [1].
>
> It's also been updated to be deployed as a snap [2], which means it's
> better confined inside its system and a bit easier for me to maintain and
> replicate.
>
> Finally, I also took this chance to move over the primary to a different
> provider and datacenter. The deployment continues to have automatic
> failover across datacenters and cities.
>
>
> Please do let me know if you observe any hiccups.
>
>
> [1] https://golang.org/x/crypto/acme/autocert
> [2] http://snapcraft.io
>
>
> gustavo @ http://niemeyer.net
>



-- 

gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] gopkg.in updated: snap, Let's Encrypt, etc

[Gustavo Niemeyer]

Hello all,

After complaints about the previous StartCom TLS certificate, it was moved
over to Let's Encrypt's dynamic generation of certificates using the
autocert package [1].

It's also been updated to be deployed as a snap [2], which means it's
better confined inside its system and a bit easier for me to maintain and
replicate.

Finally, I also took this chance to move over the primary to a different
provider and datacenter. The deployment continues to have automatic
failover across datacenters and cities.


Please do let me know if you observe any hiccups.


[1] https://golang.org/x/crypto/acme/autocert
[2] http://snapcraft.io


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Oxymoron: language spec: ``untyped boolean value''

[Gustavo Niemeyer]

It's a constant literal until it's used in a typed context.

This section explains it in detail:

https://golang.org/ref/spec#Constants

On Wed, Nov 2, 2016 at 6:09 AM, Martin Steffen 
wrote:

> Hi, in the language spec, e.g. in connection with ``type assertions'' and
> ``special forms'', like
>
> v, ok = x.(T)
>
> it's stated that it yield (in ok) an additional value which is both  untyped 
> and boolean
> (an ``untyped boolean value'').
>
> How should one interpret that? If ok behaves like a boolean, why is it 
> considered as untyped?
>
> Martin
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Adding YAML to the stdlib

[Gustavo Niemeyer]

On Fri, Sep 23, 2016 at 5:02 PM, Zachary Gershman 
wrote:

> Gustavo - it is not jus that YAML is well known, it is also widely used
> (as I even mentioned). It is a *standard *even though some may not want
> to consider it as such. If I can read xml in the stdlib why not yaml? And
> it is widely supported now but are you committed to supporting it for as
> long as golang is around?
>

Things don't need to be on the standard library for people to conveniently
benefit from them, whether widely used or not.

And I also don't need to commit my life to yaml. :)  The package is
Canonical's copyright and nicely licensed, and if we drop the ball someone
else can pick it up, as usual in open source. The fact we've been
maintaining it for years and it's widely used as you point out probably
means we're not doing such a bad job, though.


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Adding YAML to the stdlib

[Gustavo Niemeyer]

Hi Zachary,

You have already seen the thread, but for the benefit of others, Zach's
email comes from a thread raised and replied to yesterday on Twitter:

https://twitter.com/jvehent/status/778687333956587522

As I said there, the yaml spec is big, messy, and I wouldn't encourage
having the package in the distribution of Go. Something being well known
and appreciated is not a reason to have it in the standard library.

Also, there's nothing unfair about maintaining go-yaml. This was developed
years ago while porting the first projects of Canonical to Go, and is by
now widely used there, and we remain committed to supporting it. I also
receive regular fixes and contributions from the community, and nobody
seems upset to do so.

The most recent change was to replace the LGPL license by Apache, which was
well received. I was able to negotiate that based on requested from the
community, and were were able to do so due to the CLA that is requested for
contributions (ironic that most people CLA's as evil, yet it was used to
open permissions further).



On Fri, Sep 23, 2016 at 2:53 PM, Zachary Gershman 
wrote:

> Hey All,
>
> I wanted to get feedback here first before I move this over to the
> golang-dev mailing list (or maybe we even just start a change-set).  YAML
> as a spec is not the greatest and some would even describe it as "gross"
> but most if not all config files are written in some form of YAML (see
> kubernetes as a prime example).  YAML was not included in the stdlib and
> luckily for all of us the awesome go-yaml
>  emerged as the de facto standard for a
> majority of developers.
>
> Now, inclusion into the stdlib must pass a high bar
>  and not everything can / should be
> included but I believe that when you have over 1300 packages
>  depending on an outside
> library, you should at least have the discussion openly about whether it
> should be moved into the stdlib.
>
> Also, it is slightly unfair to have the expectation that the community
> should support a significant format through independent OSS work.
>



-- 

gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] mgo r2016.08.01 released

[Gustavo Niemeyer]

Hello Gophers,

A major new release of mgo was just published with relevant enhancements
and fixes:

http://blog.labix.org/2016/08/01/mgo-r2016-08-01

Please let me know if you find any issues.


gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] goxpath: An XPath 1.0 parser

[Gustavo Niemeyer]

Please note that the xmlpath package actually lives at:

http://gopkg.in/xmlpath.v2


On Mon, Jun 13, 2016 at 3:27 AM, Henrik Johansson 
wrote:

> How does it compare to https://godoc.org/launchpad.net/xmlpath?
> I have been using it with great satisfaction but it is not complete to my
> knowledge xpath feature wise.
>
>
> mån 13 juni 2016 kl 01:48 skrev Chris Trenkamp :
>
>> https://github.com/ChrisTrenkamp/goxpath
>>
>> I've created an XPath 1.0 parser.  The only thing missing is the id()
>> function for complicated reasons.  It also comes with a command-line
>> utility that works like xmlstarlet, except it only has XPath queries.
>>
>> Looking for feedback.  Let me know if you find this useful.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

gustavo @ http://niemeyer.net

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.