Joe added a comment. Apart from the concerns on a practical use case which I agree with, I have a big doubt about the implementation idea:
I am in general a fan of the paradigm that it's better to beg for forgiveness than to ask for permission, and of Postel's robustness principle, so I don't really see what use a service in front of kafka would serve us, apart from introducing another software that could fail and some latency. Messages we send onto kafka will be anyways verified on the receiving end (considering them "trusted" would be foolish), so we will need to write validation libraries in basically all the languages we will consume our data from; this is the standard way to build communications protocols and I don't see a good reason for introducing a level of indirection here. So, I have two questions I'd like an answer to: - What is the advantage of having a service validate messages before they get into the queue (Kafka or other doesn't really matter) - Why building libraries that do the validations based on shared schemas not enough? TASK DETAIL https://phabricator.wikimedia.org/T114443 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: EBernhardson, bd808, Joe, dr0ptp4kt, madhuvishy, Nuria, ori, faidon, aaron, GWicke, mobrovac, Halfak, Eevans, Ottomata, Matanya, Aklapper, JAllemandou, jkroll, Smalyshev, Hardikj, Wikidata-bugs, Jdouglas, RobH, aude, Deskana, Manybubbles, mark, JanZerebecki, RobLa-WMF, fgiunchedi, Dzahn, jeremyb, chasemp, Krenair _______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
