[Wikitech-l] commons.wikimedia.org allowing directory indexes and web robots

2009-07-18 Thread Alexandre Dulaunoy
Hi All, Commons.wikimedia.org is growing and provides a quite complete set of media files including a lot of interesting historical documents. Contributors are relying on the availability and persistence of commons.wikimedia.org but currently the full export is only available on

Re: [Wikitech-l] commons.wikimedia.org allowing directory indexes and web robots

2009-07-18 Thread David Gerard
2009/7/18 Alexandre Dulaunoy a...@foo.be: I was wondering if it would be possible to allow web robots to access http://upload.wikimedia.org/wikipedia/commons/ to gather and mirror the media files. As this is pure HTTP, the mirroring could benefit from the caching mechanisms of HTTP object

Re: [Wikitech-l] commons.wikimedia.org allowing directory indexes and web robots

2009-07-18 Thread Alexandre Dulaunoy
On Sat, Jul 18, 2009 at 3:20 PM, David Gerarddger...@gmail.com wrote: 2009/7/18 Alexandre Dulaunoy a...@foo.be: I was wondering if it would be possible to allow web robots to access http://upload.wikimedia.org/wikipedia/commons/ to gather and mirror the media files. As this is pure HTTP, the

Re: [Wikitech-l] commons.wikimedia.org allowing directory indexes and web robots

2009-07-18 Thread Robert Rohde
On Sat, Jul 18, 2009 at 6:20 AM, David Gerarddger...@gmail.com wrote: 2009/7/18 Alexandre Dulaunoy a...@foo.be: I was wondering if it would be possible to allow web robots to access http://upload.wikimedia.org/wikipedia/commons/ to gather and mirror the media files. As this is pure HTTP, the

Re: [Wikitech-l] commons.wikimedia.org allowing directory indexes and web robots

2009-07-18 Thread David Gerard
2009/7/18 Robert Rohde raro...@gmail.com: On Sat, Jul 18, 2009 at 6:20 AM, David Gerarddger...@gmail.com wrote: It'd actually be better if Google properly indexed text pages whose name ends in .jpg or whatever ... but they're aware we'd like that, so it's up to them. Which is why my

[Wikitech-l] Convert XML to HTML?

2009-07-18 Thread Chengbin Zheng
Since the static HTML Wikipedia is not updating (please update), and XML updates like everyday, the logical choice is to go with XML. Is there any way to convert XML to HTML, like the static HTML version? I need it in HTML, and I don't want a one year old version of Wikipedia, with all the useless

Re: [Wikitech-l] secure slower and slower

2009-07-18 Thread William Allen Simpson
Aryeh Gregor wrote: For what it's worth, I also don't think SSL is worthless. I don't personally see any reason to go out of my way to use it, and think it's a little odd for someone else to do that given the marginal benefit it would provide by any metric. But I'd definitely agree that if it