Ottomata added a comment. Or produce time.
But really, even if we fill in defaults during production or consumption, this will still be a problem for historical data. Data is only consumed into Hadoop once, and schema changes can happen after consumption time. If you have no way of associating a particular historical event with a schema, you won't be able to fill in defaults properly for missing keys. Avro solves this during binary deserialization, by filling in the defaults from the reader's schema. We can't avoid this problem with JSON Schema, but we can make it easier to deal with with by having a built in mapping from an event to a schema. TASK DETAIL https://phabricator.wikimedia.org/T116247 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mobrovac, Ottomata Cc: intracer, EBernhardson, Smalyshev, yuvipanda, Hardikj, daniel, aaron, GWicke, mobrovac, MZMcBride, bd808, JanZerebecki, Halfak, Krenair, brion, chasemp, Eevans, mmodell, Ottomata, Mattflaschen, Matanya, Aklapper, JAllemandou, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, RobLa-WMF, jeremyb _______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
