Ah, my apologies, I was wrong. Panicking in the midst of a func passed to one of these pipeline stages most certainly *will not* shut down the entire pipeline and cannot be recovered like I had suggested. Sorry about that.
Sincerely, K. Alex Mills On Thu, Aug 11, 2022 at 4:56 PM K. Alex Mills <k.alex.mi...@gmail.com> wrote: > See the example added in this PR > <https://github.com/splunk/go-genlib/pull/24/files#diff-be22e454f59be60b572bf51007825f39f863a2b8e5dacfb32efeab4d1177d2f9R680> > for one way to do error handling with this library without involving > panic/recover. > > One possible drawback to this approach is that it doesn't provide a way to > return an error directly from a `func` which has been passed to a pipeline > stage. I considered that, but it would seem to involve yet another pair of > variants for each proto stage: e.g. Map, MapCtx, MapCtxErr, and MapErr, > with the following signatures: > > func Map [S, T any](ctx context.Context, in <-chan S, f func(S) T, > opts ...Option) <-chan T > func MapCtx [S, T any](ctx context.Context, in <-chan S, f > func(context.Context, S) T, opts ...Option) <-chan T > func MapErr [S, T any](ctx context.Context, in <-chan S, f func(S) (T, > error), opts ...Option) <-chan T > func MapCtxErr[S, T any](ctx context.Context, in <-chan S, f > func(context.Context, S) (T, error), opts ...Option) <-chan T > > ...this proliferation of variants of essentially "the same" function is > making me reconsider some of my life choices. Asking each stage to capture > and interact with the ErrorSink yields a smaller API surface area. That > said, the Err variants would allow the library to do things like store > the ErrorSink in the context and use it if it's present. > > I'm not sure that's worth the complexity, and it feels very much like > "magic", where it seems Go "should" be explicit, but I can be convinced > otherwise. > > All that said, if you *really* want to panic and you don't mind the > pipeline shutting down whenever you do, I don't think there's anything > stopping you from doing so with this library. Just use recover at the top > of the func that spins up the pipeline and use context.WithCancel to shut > the rest of the pipeline down in that case. Let me know if that's unclear > and I'll do my best to provide an example. > > Sincerely, > > K. Alex Mills > > > On Thu, Aug 11, 2022 at 4:18 PM Robert Engels <reng...@ix.netcom.com> > wrote: > >> Every modern language with generics or similar has a streams package so >> there’s no need to call out Java. >> >> Like I said, you can chain by passing and checking the errors within the >> streamcontext. >> >> On Aug 11, 2022, at 3:59 PM, Jan Mercl <0xj...@gmail.com> wrote: >> >> >> >> >> On Thu, Aug 11, 2022, 22:37 Robert Engels <reng...@ix.netcom.com> wrote: >> >>> I don’t think that is relevant. It is very difficult to do chaining with >>> Go’s error model. >>> >> >> I don't think "Go's error model" is a thing. The language specification >> does not mention it and I'm not aware of any official Go documentation that >> should be considered the "Go's error model". >> >> Go has the predefined error interface with a single method. That's all. >> The rest is at most a convention. No need to make a ceremony of it. >> >> Wrt "it's very difficult...". Of course it's difficult to do idioms from >> other languages in Go when those idioms Go intentionally does not support. >> So the claim is sort of a tautology, isn't it? >> >> Once again, having different preferences is okay. Generalizing personal >> preferences as Go fault is IMO not okay. >> >> Maybe you can fill a proposal at the Go issue tracker to add all/some/any >> Java/etc idioms to Go and let's see how it works. >> >> You can pass a shared context to every node and store the error in the >>> context and protect against concurrent access. It’s doable but not easy. >>> >>> Map/reduce and most functional patterns are easily represented using >>> chains. >>> >>> But like I said, I would use panic/recover in the framework to make it >>> easier. >>> >>> >>> >>> On Aug 11, 2022, at 3:27 PM, Jan Mercl <0xj...@gmail.com> wrote: >>> >>> >>> >>> >>> On Thu, Aug 11, 2022, 21:36 Robert Engels <reng...@ix.netcom.com> wrote: >>> >>>> I’d say it certainly highlights a problem with Go’s error model. >>>> Exceptions would fit nicely here - instead it seems you needed to ignore >>>> all error handling - because chaining is impossible with error returns. >>>> >>> >>> It's okay if someone prefers the way Java does things, but the best >>> thing about Go is IMHO that it is not Java. And I still hope it stays away >>> from becoming Java as long as possible. >>> >>> >>>> A streams api with panic/recover is needed. >>>> >>>> On Aug 11, 2022, at 12:55 PM, K. Alex Mills <k.alex.mi...@gmail.com> >>>> wrote: >>>> >>>> >>>> Hello Gophers, >>>> >>>> I recently had an opportunity to try out Go generics on a small >>>> pipelines package >>>> <https://pkg.go.dev/github.com/splunk/go-genlib/pipelines>, along with >>>> some of my coworkers. >>>> >>>> The overall goal of this package is to provide helpers for separating >>>> concurrency from the core logic of the computation. The result was intended >>>> for I/O bound computations, and so it's likely inappropriate for managing >>>> short-lived goroutines. It takes a functional programming approach, >>>> providing helpers with familiar names seen in other APIs like Map, FlatMap, >>>> OptionMap, etc. One feature which I am particularly happy with is that >>>> concurrency concerns like worker pool size and channel buffers are >>>> configurable with minimal disruption to the rest of the code. >>>> >>>> Take a look at the library >>>> <https://pkg.go.dev/github.com/splunk/go-genlib/pipelines> and its >>>> accompanying blog post <https://kalexmills.com/journal/pipelines/>. >>>> I'm open to any of your thoughts, suggestions, and issue reports. >>>> >>>> Sincerely, >>>> >>>> K. Alex Mills >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/CALJzkY_zASs-YOukv6ciSO45b93jz39DmjAWA915kfBuwimkgQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/golang-nuts/CALJzkY_zASs-YOukv6ciSO45b93jz39DmjAWA915kfBuwimkgQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/6954C3FB-0E78-4922-8889-90FA58BA3F16%40ix.netcom.com >>>> <https://groups.google.com/d/msgid/golang-nuts/6954C3FB-0E78-4922-8889-90FA58BA3F16%40ix.netcom.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/CAA40n-WfWSMbV8p79xXGbS2%2BQ0M8pQfUPupqGnzGAdbo%2BTx0JA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/golang-nuts/CAA40n-WfWSMbV8p79xXGbS2%2BQ0M8pQfUPupqGnzGAdbo%2BTx0JA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/CAA40n-VQnNZQZ1%2BpuivmOFs-2B7TnhtPeugGPAJ%3DCKc8PVAknQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/golang-nuts/CAA40n-VQnNZQZ1%2BpuivmOFs-2B7TnhtPeugGPAJ%3DCKc8PVAknQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALJzkY8%3DX%2BAhzzHjQn-Rv4sd4B60izKd_gG4mPevMVYAKAnFYQ%40mail.gmail.com.
