On 9 September 2012 22:30, Roberto Flores <f.roberto....@gmail.com> wrote: > I think there is a slight misunderstanding on what my app is and does: > > It is an offline Wikipedia (et al) viewer that contains all content > articles in the dump. > Everything must be contained within the app's code and the processed dump > files downloadable from my own site (gearapps.com) > >> Take a look at http://en.wikipedia.org/w/api.php?action=parse... > My app is supposed to be fully offline. It does not make any network > connections, thus I cant use the online api. > I need to have the template-processing code within the app or the templates > pre-processed into the dump. > >> Also a 7GB app is something you want >> to CLEARLY state as eating up that much device space/ download >> bandwidth is probably a problem for most users > > The files are provided on my own site, so it doesn't add any load to > Wikipedia's servers. > The file sizes are viewable upon trying to download them. > > 2012/9/9 John <phoenixoverr...@gmail.com> > >> Take a look at http://en.wikipedia.org/w/api.php?action=parse it is >> exactly what you are looking for. Also a 7GB app is something you want >> to CLEARLY state as eating up that much device space/ download >> bandwidth is probably a problem for most users >> >> On Sun, Sep 9, 2012 at 3:07 PM, Roberto Flores <f.roberto....@gmail.com> >> wrote: >> > Allow me to reply to each point: >> > >> > (By the way, my offline app is called WikiGear Offline:) >> > http://itunes.apple.com/us/app/wikigear-offline/id453614487?mt=8 >> > >> >> Templates are dumped just like all other pages are... >> > >> > Yes, but that's only a text description of what the template does. >> > Code must be written to actually process them into HTML. >> > There are tens of thousands of them, and some can't be even programmed by >> > me (e.g., Wiktionary's conjugation templates) >> > If they were already pre-processed into HTML inside the articles' >> contents, >> > that would solve all of my problems. >> > >> >> what purpose would the dump serve? you dont want to keep the full dump >> >> on the device. >> > >> > I made an indexing program that selects only content articles (namespaces >> > included) and compresses it all to a reasonable size (e.g. about 7gb for >> > the English Wikipedia) >> > >> >> How would this template API function? What does import mean? >> > >> > By this I mean, a set of functions written in some computer language to >> > which I could send them the template within the wiki markup and receive >> > HTML to display. >> > >> > Wikipedia does this whenever a page is requested, but I ignore the exact >> > mechanism through which it's performed. >> > Maybe you just need to make that code publicly available, and I'll try to >> > make it work with my application somehow. >> > >> > >> > 2012/9/9 Jeremy Baron <jer...@tuxmachine.com> >> > >> >> On Sun, Sep 9, 2012 at 6:34 PM, Roberto Flores <f.roberto....@gmail.com >> > >> >> wrote: >> >> > I have developed an offline Wikipedia, Wikibooks, Wiktionary, etc. app >> >> for >> >> > the iPhone, which does a somewhat decent job at interpreting the wiki >> >> > markup into HTML. >> >> > However, there are too many templates for me to program (not to >> mention, >> >> > it's a moving target). >> >> > Without converting these templates, many articles are simply >> unreadable >> >> and >> >> > useless. >> >> >> >> Templates are dumped just like all other pages are. Have you found >> >> them in the dumps? which dump are you looking at right now? >> >> >> >> > Could you please provide HTML dumps (I mean, with the templates >> >> > pre-processed into HTML, everything else the same as now) every 3 or 4 >> >> > months? >> >> >> >> 3 or 4 month frequency seems unlikely to be useful to many people. >> >> Otherwise no comment. >> >> >> >> > Or alternatively, could you make the template API available so I could >> >> > import it in my program? >> >> >> >> How would this template API function? What does import mean? >> >> >> >> -Jeremy >> >> >> >> _______________________________________________ >> >> Wikitech-l mailing list >> >> Wikitech-l@lists.wikimedia.org >> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> >> >> > _______________________________________________ >> > Wikitech-l mailing list >> > Wikitech-l@lists.wikimedia.org >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> >> _______________________________________________ >> Wikitech-l mailing list >> Wikitech-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Basically since you are making an offline app you either need to parse the wikipages into html pages offline yourself, or include parsing code into your app. You are not the first to want this, but due to the nature and complexity of the markup, which includes "parser functions", and the parser, this is not trivial. The only parser that is guaranteed to parse MediaWiki markup is MediaWiki, but the parser is tied to other code. There is an open feature request to separate this code so apps like yours can take just the part of the rendering code you need, or translate that part of the code into another programming language. Bug 25984 - Isolate parser from database dependencies https://bugzilla.wikimedia.org/show_bug.cgi?id=25984 Nobody at WikiMedia are working on this, but there's some patches from other people that will certainly get you on your way. But the developers at WikiMedia are very busy making a whole new parser and WYSIWYG editor to go with it. Hopefully this will clean up the code to the point that making your own parser becomes a lot easier. Good luck and sympathy (-: Andrew Dunbar (hippietrail) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
