Hi! I made a a Meteor DDP API to the stream of recent changes on all WikiMedia wikis. Now you can simply use DDP.connect on in your Meteor application to connect to stream of changes on Wikipedia. You can use MongoDB queries to limit only to those changes you are interested in. If there is interest, I could add also full diffs support and then you could try to hit this API. We could probably also eventually host it on Wikimedia Labs.
http://wikimedia.meteor.com/ Mitar On Fri, Dec 12, 2014 at 11:53 PM, Maximilian Klein <isa...@gmail.com> wrote: > Hello Researchers, > > I've been playing with Recent Changes Stream Interface recently, and have > started trying to use the API's "action=compare" to look at every diff of > every wiki in real time. The goal is to produce real-time analytics on the > content that's being added or deleted. The only problem is that is will > really hammer the API with lots of reads since it doesn't have a batch > interface. Can I spawn multiple network threads and do 10+ reads per second > forever without the API complaining? Can I warn someone about this and get a > special exemption for research purposes? > > The other thing to do would be to use "action=query" to get the revisions in > batches and do the diffing myself, but then i'm not guaranteed to be diffing > in the same way that the site is. > > What techniques would you recommend? > > > Make a great day, > Max Klein ‽ http://notconfusing.com/ > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > -- http://mitar.tnode.com/ https://twitter.com/mitar_m _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
