Re: [Wiki-research-l] Request for feedback on new data dump formats

2011-03-31 Thread Aaron Halfaker
Yes. That's a lot like what I had in mind. On Thu, Mar 31, 2011 at 7:33 PM, Yuvi Panda wrote: > Would incremental dumps, as described by brion long time ago > (http://leuksman.com/log/2007/10/14/incremental-dumps/) be what you're > looking for? > > On Fri, Apr 1, 2011 at 5:01 AM, Aaron Halfaker

Re: [Wiki-research-l] Request for feedback on new data dump formats

2011-03-31 Thread Yuvi Panda
Would incremental dumps, as described by brion long time ago (http://leuksman.com/log/2007/10/14/incremental-dumps/) be what you're looking for? On Fri, Apr 1, 2011 at 5:01 AM, Aaron Halfaker wrote: > If periodic update dumps are being considered, information that describes > changes to old data

Re: [Wiki-research-l] Request for feedback on new data dump formats

2011-03-31 Thread Aaron Halfaker
If periodic update dumps are being considered, information that describes changes to old data (page deletes, user renames, etc) would be very useful to have along with new revisions. -Aaron On Mar 31, 2011 6:27 PM, "Luca de Alfaro" wrote: > I think I would be very interested in 3, or even, in hav

Re: [Wiki-research-l] Request for feedback on new data dump formats

2011-03-31 Thread Luca de Alfaro
I think I would be very interested in 3, or even, in having every month a dump of that month's revisions. As I have built tools for the xml dumps, no change in format is good for me (and for WikiTrust). I would find incremental dumps (with occasional, yearly, full dumps) much easier to manage tha

[Wiki-research-l] Request for feedback on new data dump formats

2011-03-31 Thread Yuvi Panda
Hi, I'm a student planning on doing GSoC this year on mediawiki. Specifically, I'd like to work on data dumps. I'm writing this to gauge what would be useful to the research community. Several ideas thrown about include: 1. JSON Dumps 2. Sqlite Dumps 3. Daily dumps of revisions in last 24 hours 4.