Marten, Given they want to use `persistent_term` for performance, I think that the fallback probably wouldn't qualify.
If this was a normal library, we could have a compile-time setting to choose an option and it would just recompile on a change. Then we could implement it such that it had no runtime impact. However, since this is the standard library, I believe that such recompilation is off the table. (Also, sorry I didn't respond sooner.) -J On Friday, November 6, 2020 at 7:24:53 AM UTC-8 w...@resilia.nl wrote: > One possibility to allow this and other features waiting on > `persistent_term` already on OTP 21.0+ would be to provide a non-NIF > fallback implementation. > After all, `persistent_term is based on userspace-libraries written in > Elixir and Erlang that were released before (c.f. the FastGlobal Elixir > library on Hex and mochiglobal > <https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl> (part > of the mochiweb Erlang library).) > > Very nice to see steps being made to support inspect redaction! > > ~Marten > On 06-11-2020 15:30, José Valim wrote: > > After a brief discussion with the Elixir team, we are inclined to go more > with option 2. However, in order to make it efficient, we want to use > persistent_term, which is only available on 21.3 and Elixir currently > supports 21.0+. > > We will drop Erlang/OTP 21 support once Erlang/OTP 24 is out but it is > unclear if that will be before Elixir v1.12. Alternatively we can require > Erlang/OTP 21.3. In any case, I have added an item to track this here: > https://github.com/elixir-lang/elixir/issues/8414 > > Thank you! > > On Fri, Nov 6, 2020 at 10:25 AM Allen Wyma <allen...@gmail.com> wrote: > >> Looks like they just use the derive functionality from inspect: "@derive >> {Inspect, except: @ecto_redact_fields}" so you can also look into using >> that. If it's for struct outside of your control, that may be more difficult >> >> On Fri, Nov 6, 2020 at 5:21 PM 'Jayson Vantuyl' via elixir-lang-core < >> elixir-l...@googlegroups.com> wrote: >> >>> That would help with Ecto. Unfortunately we still see a bunch of stuff >>> from the likes of Elixir GRPC and various structs from libraries that don't >>> have that kind of functionality. We were kind of hoping for a top-down, >>> whitelist approach. We figure that it's easier to plug the leak from the >>> top (*inspect/2* itself) than it is to try to deal with all of the >>> leaks at the bottom (everything that tries to use it). >>> >>> On Friday, November 6, 2020 at 1:15:37 AM UTC-8 allen...@gmail.com >>> wrote: >>> >>>> I believe that the new changeset "redact" attribute should help with >>>> this? >>>> >>>> On Fri, Nov 6, 2020 at 4:56 PM 'Jayson Vantuyl' via elixir-lang-core < >>>> elixir-l...@googlegroups.com> wrote: >>>> >>>>> Hey everybody, >>>>> >>>>> *Use Case* >>>>> >>>>> I have what I think is actually a pretty common problem. All over my >>>>> code base, there are uses of *inspect/2*. This is a wonderful thing >>>>> that helps with debugging. It is less wonderful when it spews PII all >>>>> over >>>>> your logs / error pages and you find yourself having sent somebody's >>>>> social >>>>> security number to DataDog or disclosed their home address on a 500 page. >>>>> >>>>> Then your lawyer starts doing that thing where their eye twitches, you >>>>> need >>>>> to notify four different regulators on three continents, alert all of >>>>> your >>>>> customers with scary messages, are made to attend 3-hours of mandatory >>>>> re-education wherein you learn to recite the GDPR from memory, and >>>>> eventually sacrifice three interns to appease the compliance gods. >>>>> >>>>> *Temporary Hack* >>>>> >>>>> What myself and some co-conspirators have done to address this is >>>>> overriding the *Inspect* protocol for the built-in types to redact >>>>> things by default and then have a whitelist for the bits we want to show. >>>>> >>>>> Given that our top-level logging metadata is a map, we can pretty much >>>>> just >>>>> whitelist keys there and call it a day. This works fabulously, but it >>>>> violates Erlang's expectations significantly. While Erlang is probably >>>>> used to that, it gets quite cross and refuses to generate a release >>>>> because >>>>> it doesn't have a good way to know which .beam file to put in the release. >>>>> >>>>> As you might imagine, I'm pretty bummed that I can't use releases and >>>>> have to ignore tons of very alarming sounding warnings about redefining >>>>> modules. >>>>> >>>>> *Options* >>>>> >>>>> Could we consider some solutions to make redaction require less >>>>> unforgivable sins against code loading? To start, three directions have >>>>> been proposed by the various people I've talked to: >>>>> >>>>> >>>>> 1. Instead of implementing *Inspect* for the built-in types, do >>>>> that inspection in a handler for *Any*; thereby allowing >>>>> overriding of the built-in types easily. >>>>> 2. Wedge something into a common entry point (maybe in >>>>> *Inspect.Algebra*?) allows us to specify a global redaction >>>>> function. Perhaps configure this with a global config value? >>>>> 3. Implement some sort of overriding layer for just the *Inspect* >>>>> protocol. >>>>> >>>>> >>>>> In terms of pros and cons, for #1... >>>>> >>>>> - *Pro:* Works well for built-ins. >>>>> - *Pro:* Implementing this is very straightforward. >>>>> - *Pro:* This probably doesn't break any existing code, very small >>>>> blast radius. >>>>> - *Con:* Doesn't work at all as soon as anything defines its own >>>>> inspection protocol. >>>>> - *Con:* Isn't amenable to configuration at runtime (maybe this is >>>>> not an issue?). >>>>> >>>>> As for #2... >>>>> >>>>> - *Pro:* *Can* be configured at runtime. >>>>> - *Pro:* I have no idea how to implement this and *Inspect.Algebra* >>>>> scares me. >>>>> - *Pro:* This probably doesn't break any existing code, very small >>>>> blast radius. >>>>> - *Con:* Given that the Inspect protocol is pretty much "turn X >>>>> into string", I'm not sure how much redaction we could really do if we >>>>> allow the existing protocol to run. >>>>> >>>>> As for #3... >>>>> >>>>> - *Pro:* This provides a clear way to just replace the protocol >>>>> for a given type. >>>>> - *Pro:* Implementing this is very straightforward. >>>>> - *Pro: *This probably doesn't break any existing code, very small >>>>> blast radius. >>>>> - *Con:* It's all fun and games until a library does it, then you >>>>> need Override2 or 3 or 4... >>>>> - *Con:* Probably gets redundant if there ever is a blessed way to >>>>> override protocols. >>>>> - *Con:* I already pitched this to a few Elixir celebrities and >>>>> they thought it was a bit too hacky. >>>>> >>>>> *In Closing* >>>>> >>>>> So, yeah, in the long term, maybe we'll have a blessed way to override >>>>> protocols; but, short of that, there's got to be some stopgap, right? >>>>> What >>>>> do people think of adopting something like one of these approaches so >>>>> that >>>>> my PII problems evaporate and I can finally build some sweet, sweet >>>>> releases? I'll even implement it myself! I promise! >>>>> >>>>> Thanks, >>>>> >>>>> - J >>>>> >>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elixir-lang-core" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to elixir-lang-co...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elixir-lang-core/d5d9a932-930c-46f7-a3fb-2f1a9ae06112n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/elixir-lang-core/d5d9a932-930c-46f7-a3fb-2f1a9ae06112n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elixir-lang-co...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/38307a23-2568-4c48-ae8c-76cb70e7541bn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elixir-lang-core/38307a23-2568-4c48-ae8c-76cb70e7541bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-co...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/CAJuWSyL8xhXbZf0ca_nZ4-dJU_Xej3UJ_8-dQFJp0%2B9XX1znpw%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elixir-lang-core/CAJuWSyL8xhXbZf0ca_nZ4-dJU_Xej3UJ_8-dQFJp0%2B9XX1znpw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-co...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L_CuMDq2JVLz2tYog33eTBLR_YF36B-c_DY5VUaVQNsQ%40mail.gmail.com > > <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L_CuMDq2JVLz2tYog33eTBLR_YF36B-c_DY5VUaVQNsQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/714fa520-ec27-44fe-9781-dc3b580586a2n%40googlegroups.com.
