> So having a “meta/relaxed decoder” usually leads to specification/interoperability/security problems down the road I respectfully disagree. Since it's only relaxed with regard to decoding, it follows the robustness principle where you be liberal in what you accept.
Within a system, the encoding should be explicitly defined, but when that system has to consume base64 data from outside, being liberal actually avoids interoperability problems. On Wednesday, February 3, 2021 at 1:10:32 PM UTC+8 ren...@ix.netcom.com wrote: > Sorry it just doesn’t “feel right”. There are different encoding scheme as > laid out in the RFC. and other RFCs > <https://en.wikipedia.org/wiki/Base64#Variants_summary_table> that cover > their uses. > > If you have a system that states “send us Base64 data” it is poorly > specified - better to state, send us Base64 data according to RFC 4648 > base64url format or according to RFC-2045. > > In fact, the RFC states: > > "This encoding may be referred to as "base64url". This encoding > should not be regarded as the same as the "base64" encoding and > should not be referred to as only "base64". Unless clarified > otherwise, "base64" refers to the base 64 in the previous section.” > > It also states: > > "If non-alphabet characters are ignored, instead of causing rejection > of the entire encoding (as recommended), a covert channel that can be > used to "leak" information is made possible." > > So having a “meta/relaxed decoder” usually leads to > specification/interoperability/security problems down the road. I realize > that in the “real world” you are often forced to interoperate with these > “bad” systems, but as most things in Go, better to be explicit and report > errors rather than be clever. > > > > On Feb 2, 2021, at 12:34 PM, 'Axel Wagner' via golang-nuts < > golan...@googlegroups.com> wrote: > > On Tue, Feb 2, 2021 at 6:43 PM Robert Engels <ren...@ix.netcom.com> wrote: > >> What “padding” are you referring to? Each must be 2 characters. And there >> is a standard that covers this https://tools.ietf.org/html/rfc4648 >> > > Yes, there indeed is. Section 5 describes a second encoding scheme, used > for URLs and the like. Section 3.2 also talks about the padding I'm > referring to (it's defined elsewhere in the standard) and mentions that, in > certain situations, it can be omitted. In particular, you can omit padding > and, in the decoder, implicitly pad to a multiple of 4 bytes. > > I don't really understand what's the argument is here. The question was if > it is possible to handle all four encoding schemas > <https://golang.org/pkg/encoding/base64/#pkg-variables> supported by the > Go base64 package in one swoop, because as-is, the API requires you to pick > one schema and just see if it returns an error. Roger provided, IMO, a > pretty good answer to that: You can wrap the io.Reader in one that > transparently rewrites any of the four into one well-known one, which can > then be handled by the corresponding decoder. His link provides the code > for an implementation of such a reader. > > >> >> On Feb 2, 2021, at 10:57 AM, 'Axel Wagner' via golang-nuts < >> golan...@googlegroups.com> wrote: >> >> >> Rogers approach seems like the best one to me - wrap the input in a >> custom `io.Reader` that transparently replaces `-_` with `+/` respectively >> (and drop trailing `=`). The bufio approach doesn't work, because there is >> no guarantee that one of the distinguishing characters is early in the >> stream and the "send it to multiple decoders" approach duplicates effort >> and wastes resources. >> >> On Tue, Feb 2, 2021 at 5:43 PM Amnon <amn...@gmail.com> wrote: >> >>> Reading through a bufio.Reader is often useful for these situations. >>> You can peek the beginning of the input in order to decide which decoder >>> to use. >>> >>> Another option is to use the io.TeeReader to duplicate the reader, >>> and then send one copy to each decoder. >>> One will succeed, and give you the output. >>> But you will need to drain the one that fails to prevent the TeeReader >>> form stalling. >>> >>> >>> On Tuesday, 2 February 2021 at 14:17:19 UTC axel.wa...@googlemail.com >>> wrote: >>> >>>> This question isn't about the decoded data, but about *which* base64 >>>> format is used - i.e. if it uses padding or not and what 2 characters are >>>> used outside of a-zA-Z0-9. The most common ones use +/ and -_, so it's >>>> easy >>>> to tell which is used and just accept either (and padding can be viewed as >>>> optional during decoding anyway). >>>> >>>> On Tue, Feb 2, 2021 at 2:37 PM Robert Engels <ren...@ix.netcom.com> >>>> wrote: >>>> >>>>> Base64 is always ASCII. The encoded data may be in an arbitrary >>>>> format. You need to pass additional metadata or try and detect its >>>>> encoding. >>>>> >>>>> On Feb 2, 2021, at 6:50 AM, roger peppe <rogp...@gmail.com> wrote: >>>>> >>>>> >>>>> In case you find it helpful, here's a clone of the base64 command that >>>>> I wrote in Go. I did it precisely because I wanted to be able to decode >>>>> any >>>>> encoding scheme interchangeably. >>>>> >>>>> https://github.com/rogpeppe/misc/blob/master/cmd/base64/base64.go >>>>> >>>>> I agree that it might be useful to have some of this functionality >>>>> available in the standard library. >>>>> >>>>> cheers, >>>>> rog. >>>>> >>>>> On Tue, 2 Feb 2021 at 09:08, hey...@gmail.com <hey...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have an io.Reader whose content is encoded in base64 with encoding >>>>>> type unknown. Since there shouldn't be any ambiguity between the two, is >>>>>> it >>>>>> possible to make the base64 automatically pick the right one to decode? >>>>>> >>>>>> Currently I have to read everything out to pin down the encoding, >>>>>> which defeats the purpose of using an io.Reader. >>>>>> >>>>>> Is there a solution to this problem? >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "golang-nuts" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/golang-nuts/0ccee37d-319e-41b3-9bfd-3dc46e0fad78n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/golang-nuts/0ccee37d-319e-41b3-9bfd-3dc46e0fad78n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/CAJhgacjkUUSr-dOPFU-W4vG_AXZRY_dYYe2ti-iPuu_XUL%2BNVw%40mail.gmail.com >>>>> >>>>> <https://groups.google.com/d/msgid/golang-nuts/CAJhgacjkUUSr-dOPFU-W4vG_AXZRY_dYYe2ti-iPuu_XUL%2BNVw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/CF76EF37-6F82-42FF-B4D6-4B9FC02F25FC%40ix.netcom.com >>>>> >>>>> <https://groups.google.com/d/msgid/golang-nuts/CF76EF37-6F82-42FF-B4D6-4B9FC02F25FC%40ix.netcom.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/44621cb2-fae7-4f40-9e50-35b157f4e838n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/golang-nuts/44621cb2-fae7-4f40-9e50-35b157f4e838n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGGkOX9HzF6e178Cu6BD9Hg4x5LT7AutHkUDPvyK%2BFmYg%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGGkOX9HzF6e178Cu6BD9Hg4x5LT7AutHkUDPvyK%2BFmYg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGwDKwoi4TisXPwi0-85%3D_yOqY1AOBw%3DCnH%3D7E4jhJuBg%40mail.gmail.com > > <https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGwDKwoi4TisXPwi0-85%3D_yOqY1AOBw%3DCnH%3D7E4jhJuBg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5963b370-e82f-4f50-9d3a-037f3899b5e8n%40googlegroups.com.
