I find closed keys specs generally easier to work this (e.g. it protects from cases when I forget to spec some new data returned from the function). In my current project the number of open keys specs is ~1.5% and they're are for entities where the shape is really unknown beforehand. At the moment my rule of thumb is to start with strictest specs possible and then loosen them based on the feedback from code.
So far I can't recall to have any painful breakages described in Speculation talk. I haven't analyzed why, maybe eventually I'll stumble upon such case or this may be due to the fact that I have control over all the APIs and consumer code in the project. Here's my take at adding `:disallowed-keys` into the `explain-data` of the closed spec (see `speced-keys` macro): https://gist.github.com/metametadata/53a847cd3b02056e8e4c124e63d9ae5a: (s/def ::req-field1 any?) (let [s (sp/speced-keys :req [::req-field1]) value {::req-field1 123 :extra1 100} ; act actual (s/explain-data s value)] ; assert (is (= {::s/problems [{:in [] :path [] :pred 'no-disallowed-keys? :disallowed-keys #{:extra1} ; <------------- :val value :via []}] ::s/spec s ::s/value value} actual))) As a bonus, `speced-keys` specs can be merged via `merge-keys`. They also validate at runtime that all fields have specs already registered (related topic: https://groups.google.com/forum/#!topic/clojure/i8Rz-AnCoa8). On Monday, March 4, 2019 at 3:30:00 AM UTC+2, Daniel Dinnyes wrote: > > Hi Everyone, > > I am using spec for a while (currently on the version shipped with Clojure > 1.10). This post is intended to be a kind-of status report, focusing on a > particular issue I am facing at the moment. To explain it in detail, please > have a look on this following macro I wrote, which is an upgraded version > of *spec/keys*: > > (ns myns > (:refer-clojure :exclude [keys]) > (:require > [clojure.set :as set] > [clojure.core :as core] > [clojure.spec.alpha :as spec])) > > (defmacro keys > "Same as `clojure.spec/keys`, but accepts additional boolean option > :additional-keys. Unless :additional-keys is set true, only the declared > keys are allowed, and any additional keys will be invalid." > [& {:keys [additional-keys] :as args}] > (let [args (dissoc args :additional-keys)] > (if additional-keys > `(spec/keys ~@(mapcat identity args)) > (let [allowed-keys #{} > allowed-keys > (reduce > (fn [acc k] (into acc (k args))) > allowed-keys [:req :opt]) > allowed-keys > (reduce > (fn [acc k] (into acc (map (comp keyword name) (k args)))) > allowed-keys [:req-un :opt-un])] > `(spec/and > (spec/keys ~@(mapcat identity args)) > (fn [m#] (set/subset? (core/keys m#) ~allowed-keys))))))) > > I would like to explain my use-case for writing this, and the reason I > think there is a need for such feature. > I am writing an import/export library from a serialized XML data format, > into in-memory representation. I need the import and export functions to be > invertible: > * given an original external XML data, if I import it, and then > immediately export it, the re-exported data has to be equal to the original. > * given an original internal in-memory data, if I export it, and then > immediately import it, the re-imported data has to be equal to the original. > > The main issue is that silent data-loss is absolutely unacceptable, (e.g. > if a newer version of the XML data format has additional unspecified > fields, these additional fields will fail to be imported to the in-memory > model. Also, if any additional unspecified in-memory data gets added, it > will fail to be exported. All this would all happen silently, without any > error/warning). > > The problem I am facing is I think one of the valid use-case, spec should > have support for such scenarios. > > Another similar use-case is when due to GDPR > <https://en.wikipedia.org/wiki/General_Data_Protection_Regulation> > regulatory rules, extreme care should be taken what information gets > stored, and any unverified data is potentially violating requirements. > (e.g. a user might use clear-text comment fields to store credit-card > information, N.I. numbers, etc.) > > The original keys spec allows for additional data to be present, which is > in line with what Rich described as design goals for spec (i.e. "requiring > less", or "providing more" is not breakage but growth, and should be > welcomed, as per the "Speculation > <https://www.youtube.com/watch?v=oyLBGkS5ICk>" talk). > > After I started using the above macro, it became obvious that it isn't > versatile enough solution to the problem, because I would rather like to be > notified of additional-keys, and then decide on a case-by-case basis, if in > the given context that is to be considered an error. Instead the macro only > gives me the option to decide this at the time when defining the spec, > rather at the point of data validation. > > If I recall correctly, in the "Maybe Not > <https://www.youtube.com/watch?v=YR5WdGrpoug>" talk Rich was talking > about the concerns, that validating required and optional fields should be > separated from the shape/schema of a spec, to be a'la carte, and decided at > the point of validation, not at the time of defining the specs. It feels to > me this concern is quite similar/related issue to the one I am writing > about now. > > I have also noticed that my usage pattern is that I rarely use > *spec/valid?*, Instead prefer to use something like: > > (if-let [explanation (explain-data :my/spec data)] > (throw (ex-info "Invalid stuff" explanation) > (do-business-logic-with (spec/conform :my/spec data))) > > Due to this usage pattern, I am quite reliant on the fact that > *spec/valid?* returns *true* iff > <https://en.wikipedia.org/wiki/If_and_only_if> *spec/explain-data* > returns *nil*. > > Nevertheless, I was thinking if *spec/explain-data* would return any > unspecified keys found for a keys spec, under an *:unspecified* key in > the explain-data map that would address the issue much better than my > barbaric butchery with macros. > > I would be interested to hear what you guys think about all the above, or > if there are better workarounds/recommendations for my use-cases. > > Cheers, > Daniel > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
