Re: [Wikitech-l] Uploads on small wikis
On Sun, Apr 4, 2010 at 2:44 AM, Roan Kattouw roan.katt...@gmail.com wrote: I'm all for this: if a feature is disabled locally and supposed to be used elsewhere, just hiding the feature isn't very user-friendly. Providing a clear explanation of how this stuff works and why clearly wins. This is something that should IMO be fairly uncontroversial (we're not changing the status quo, just documenting it) and can be implemented quickly without having to wait for the discussion on how this should be done in a perfect world to conclude. Roan Kattouw (Catrope) Perhaps make the message that appears customizable (Mediawiki: namespace) that way they can set it to say anything they like (For example, explaining their restrictions and that they need to upload at commons instead.) -Peachey ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Thanks for all the responses. I think I understand the different
arguments. I'd like us to find a reasonable interim solution,
especially for small wikis like the Swahili Wikipedia, where the
upload link is now hidden for new users, and visiting the upload page
(if you do get a link from somewhere) results in an obscure error
message.
How about this as an interim fix:
Instead of hiding the upload link and showing an obscure error message
to new users, we show the link to all logged in users, and show a more
informative message to non-autoconfirmed users, something like:
Please visit the upload form on Wikimedia Commons [link to localized
version] to upload files, then return here to insert them. Uploading
to Wikimedia Commons makes the file available to all Wikimedia
languages and projects. Files can be uploaded directly to {{SITENAME}}
by more experienced users.
This could be an interim solution until we have nicer Commons
integration. Thoughts?
--
Erik Möller
Deputy Director, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 5:42 PM, Thomas Dalton thomas.dal...@gmail.com wrote: If these are projects with active users, isn't this a decision to be made by those active users rather than wikitech-l? Wikimedia defaults are decided by wikitech-l/Wikimedia. Specific communities can choose to opt out of those defaults if they choose. Requiring all communities to explicitly opt in to any changes would make the defaults almost impossible to change. The change requiring autoconfirmed for upload wasn't approved by all the communities individually in any case, so it wouldn't make sense to require the change back to be so approved. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 14 March 2010 00:34, Aryeh Gregor simetrical+wikil...@gmail.com wrote: On Fri, Mar 12, 2010 at 5:42 PM, Thomas Dalton thomas.dal...@gmail.com wrote: If these are projects with active users, isn't this a decision to be made by those active users rather than wikitech-l? Wikimedia defaults are decided by wikitech-l/Wikimedia. Specific communities can choose to opt out of those defaults if they choose. Requiring all communities to explicitly opt in to any changes would make the defaults almost impossible to change. The change requiring autoconfirmed for upload wasn't approved by all the communities individually in any case, so it wouldn't make sense to require the change back to be so approved. I wouldn't count a change to just certain projects as being a change to the defaults. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 11 Mar 2010 at 11:06, Erik Moeller wrote: Hello all, together with Frank Schulenburg and Naoko Komura, I just participated in a video-conference with the winners of the Google Kiswahili Wikipedia challenge ( http://www.google.com/events/kiswahili-wiki/ ), and we talked about some of the challenges they encountered when contributing to the Swahili Wikipedia. One of the issues was that it was very hard for them to upload files. Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. From a user experience standpoint, this is horrible. For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Thanks, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Erik, [2c opinion] To me it doesn't make sense. Commons is the home for files that are in the public domain, as that enables crosswiki use. Why would we want to change that message, and in fact work against that message? Some could argue that we would be starting to embed a counter- culture that we know that some will be resistant to change. To me it would seem far more practicable to put efforts into having images seamlessly be able to be uploaded to Commons, and provide the interface at the wiki, or direct them to those resources at Commons. Alternatively if you do allow uploads for all, and then have an automated process in place to move the images to Commons. The maintenance task of moving images to Commons from the wikis is quite significant, and I am not sure why we would be wanting to add to that. If administrators have time, it should be for valued tasks, not for make work, for decisions that seem not to pay heed to our own history, and current events. In fact, it would be interesting to see some of the data of images on small and large wikis, including the numbers transferred from each to Commons, and who undertook such actions (local vs Commons users). Regards, Andrew ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Hoi, How would Commons work for someone who speaks and reads Mazanderani, Nepali or Yoruba and does not speak English ? They are very welcome to use Commons but Commons is hardly usable if you are not able to navigate its data and procedures. I proposed less complicated procedures for the smaller Wikipedia combined with fewer options. This in order to make the use of pictures less difficult. I also proposed that people mark images that conform to Commons standards. They can be easily moved by bot. In my opinion, my proposal makes it easier on the small projects and includes a process of normalising towards the use / involvement of Commons. Thanks, GerardM On 12 March 2010 11:39, Billinghurst billinghu...@gmail.com wrote: On 11 Mar 2010 at 11:06, Erik Moeller wrote: Hello all, together with Frank Schulenburg and Naoko Komura, I just participated in a video-conference with the winners of the Google Kiswahili Wikipedia challenge ( http://www.google.com/events/kiswahili-wiki/ ), and we talked about some of the challenges they encountered when contributing to the Swahili Wikipedia. One of the issues was that it was very hard for them to upload files. Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. From a user experience standpoint, this is horrible. For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Thanks, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Erik, [2c opinion] To me it doesn't make sense. Commons is the home for files that are in the public domain, as that enables crosswiki use. Why would we want to change that message, and in fact work against that message? Some could argue that we would be starting to embed a counter- culture that we know that some will be resistant to change. To me it would seem far more practicable to put efforts into having images seamlessly be able to be uploaded to Commons, and provide the interface at the wiki, or direct them to those resources at Commons. Alternatively if you do allow uploads for all, and then have an automated process in place to move the images to Commons. The maintenance task of moving images to Commons from the wikis is quite significant, and I am not sure why we would be wanting to add to that. If administrators have time, it should be for valued tasks, not for make work, for decisions that seem not to pay heed to our own history, and current events. In fact, it would be interesting to see some of the data of images on small and large wikis, including the numbers transferred from each to Commons, and who undertook such actions (local vs Commons users). Regards, Andrew ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 5:51 AM, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi, How would Commons work for someone who speaks and reads Mazanderani, Nepali or Yoruba and does not speak English ? They are very welcome to use Commons but Commons is hardly usable if you are not able to navigate its data and procedures. I proposed less complicated procedures for the smaller Wikipedia combined with fewer options. This in order to make the use of pictures less difficult. I also proposed that people mark images that conform to Commons standards. They can be easily moved by bot. In my opinion, my proposal makes it easier on the small projects and includes a process of normalising towards the use / involvement of Commons. Thanks, GerardM On 12 March 2010 11:39, Billinghurst billinghu...@gmail.com wrote: On 11 Mar 2010 at 11:06, Erik Moeller wrote: Hello all, together with Frank Schulenburg and Naoko Komura, I just participated in a video-conference with the winners of the Google Kiswahili Wikipedia challenge ( http://www.google.com/events/kiswahili-wiki/ ), and we talked about some of the challenges they encountered when contributing to the Swahili Wikipedia. One of the issues was that it was very hard for them to upload files. Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. From a user experience standpoint, this is horrible. For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Thanks, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Erik, [2c opinion] To me it doesn't make sense. Commons is the home for files that are in the public domain, as that enables crosswiki use. Why would we want to change that message, and in fact work against that message? Some could argue that we would be starting to embed a counter- culture that we know that some will be resistant to change. To me it would seem far more practicable to put efforts into having images seamlessly be able to be uploaded to Commons, and provide the interface at the wiki, or direct them to those resources at Commons. Alternatively if you do allow uploads for all, and then have an automated process in place to move the images to Commons. The maintenance task of moving images to Commons from the wikis is quite significant, and I am not sure why we would be wanting to add to that. If administrators have time, it should be for valued tasks, not for make work, for decisions that seem not to pay heed to our own history, and current events. In fact, it would be interesting to see some of the data of images on small and large wikis, including the numbers transferred from each to Commons, and who undertook such actions (local vs Commons users). Regards, Andrew ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l Really what needs doing is the ability to upload files to commons while never leaving your local wiki. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 3/11/10, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi, I am really happy that we want to help the smaller Wikipedias with uploading their pictures. It helps when it is made easy. One of the ways in which we can make it easy is to reduce the number of choices involved. When the only choice is cc-by-sa, we only need to explain one license. When the software involved is properly localised in their language, confusion will be a lot less. One of the ways that makes a big difference to people is, when they know that a small number of messages provide exactl functionality.. The WikiReader and the mobile Wikimedia experience point in that direction. There are however two observations; it produces overhead at translatewiki.net and it helps when you target people who are actively translating for that language. What you can do is make the local upload facility available once these limited number of messages have been localised. In this way it is a reward and not depended on an arbitrary number. I think that's a bad proposal, in at least two ways: 1. It does not answer the _reason_ for this restriction. The problem is people uploading many copyright violations. Having the interface in the own language is not going to solve that problem, as is shown by the simple fact that this policy was started based on experience in English and other large Wikipedias, which have had full localization for years 2. The reward that you mention does not exist: Getting autoconfirmed is in general a lesser issue than translating a large part of the interface, and furthermore the first will happen automatically when one is active, the second can only be done on a special place that a random newbie will not be looking for or stumbling upon. If we're going to make this dependant on something at all, it should be the presence on active sysops, who are willing to check for copyright violations. However, personally I would prefer some people from Commons and/or Stewards taking such a role for wikis which do not have such sysops. Localizing this part of the interface is definitely a useful thing to do, but its connection with the problem at hand is very tenuous. -- André Engels, andreeng...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
* Dmitriy Sintsov ques...@rambler.ru [Fri, 12 Mar 2010 14:57:39 +0300]: I always wondered, why to have a multilanguage wiki one has to setup a wiki farm with interlinks? Why wiki cannot be multilingual by default? By the way, MediaWiki has the preliminary support of that back in 1.10 when I've started to use it for local projects. It's a global variable $wgLanguageCode and the corresponding request parameter uselang=langcode. Why not to use the correct source language code when transferring a user from small local wiki to commons, so the links to commons will have an appropriate uselang=code parameter. Dmitriy Hmm.. it seems that commons already uses uselang parameter. It just should use it persistantly and the parameter should be passed via the links from small wikis. Dmitriy ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Casey Brown hett schreven: On Fri, Mar 12, 2010 at 7:02 AM, Dmitriy Sintsov ques...@rambler.ru wrote: Hmm.. it seems that commons already uses uselang parameter. It just should use it persistantly and the parameter should be passed via the links from small wikis. You can change the language of the Commons interface in your preferences, instead of using the ?uselang= parameter all the time. Imagine a user who does not speak English (you can simulate that by changing your interface language to a language you don't know). Further imagine the user is not familiar with Mediawiki (you can simulate that by changing your skin to a skin you never used before). I changed my interface to Arabic and used the Chick skin. I was completely lost. No chance to find the right menu to change the language. As all new users nowadays have SUL accounts, perhaps the default language on Commons for SUL users shouldn't be English, but the language selected at the SUL home wiki (just the default, it should of course still be changeable in the preferences). For unregistered users registering on Commons there should be a language selection menu. There already is a language selection menu, but it only changes the signup form and doesn't propagate to the preferences. It should propagate to the preferences! Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/12 Marcus Buck w...@marcusbuck.org: Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Hoi, That is a great argument why you cannot do it in the current configuration. It does not mean that it is desirable. It seems therefore that you want to change the current configuration. Thanks, GerardM On 12 March 2010 14:44, Roan Kattouw roan.katt...@gmail.com wrote: 2010/3/12 Marcus Buck w...@marcusbuck.org: Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 8:44 AM, Roan Kattouw roan.katt...@gmail.com wrote: 2010/3/12 Marcus Buck w...@marcusbuck.org: Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l Making uselang persist in $_SESSION wouldn't be a bad idea though (see bug 4125) -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Roan Kattouw hett schreven: 2010/3/12 Marcus Buck w...@marcusbuck.org: Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. Roan Kattouw (Catrope) Can you please elaborate? And feel free to use technical terms ;-) Why would that be a problem? We can cache the English pages so why can't we cache non-English pages? Of course the amount of rendering events will rise, but I cannot imagine why this rise would be so immense we cannot handle it. Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/11/2010 2:06 PM, Erik Moeller wrote: Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. Please don't! Just add the link to Commons, which has a *far* superior upload process, and people around who actually know what's going on. With automatic account creation, users will have a much better experience with this process than in the past, and the Commons community is constantly trying to improve the experience of actually uploading files on Commons -- but those improvements are on Commons, not xxwiki. Really, sending them to Commons is a much better solution. - -Mike -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkuaSZYACgkQst0AR/DaKHvMLACgut8HcRYIb9cCnstPjoq6y3Vg c+kAni3gmzGzsrrxqs289GN4Hxr9WMLY =xg4z -END PGP SIGNATURE- ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/12 Gerard Meijssen gerard.meijs...@gmail.com: Hoi, That is a great argument why you cannot do it in the current configuration. It does not mean that it is desirable. It seems therefore that you want to change the current configuration. This is not a question of flipping a switch, it's an argument why it's fundamentally hard to respect Accept-Language for anonymous users. For the same reason, making uselang persist in $_SESSION also doesn't work for anonymous users. For logged-in users, though, both should be fine. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Roan Kattouw schreef: 2010/3/12 Marcus Buck w...@marcusbuck.org: Additionally why do we always present English as default to unregistered users on Commons? Why don't we observe the accept-language HTTP header? That would be some easy steps to make Commons much more accessible to non-English people. That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. I understand this makes caching more difficult, but did anyone ever do measurements? Without decent metrics this is just wild guessing. Things to measure for example: * Total text requests for all sites * Total text requests for just Commons * Total misses for all sites * Total misses for just Commons I'm sure someone can dig up these metrics. Worst case all text requests for Commons go to the backend directly, right? Maarten ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/12 Marcus Buck w...@marcusbuck.org: Can you please elaborate? And feel free to use technical terms ;-) Why would that be a problem? We can cache the English pages so why can't we cache non-English pages? Of course the amount of rendering events will rise, but I cannot imagine why this rise would be so immense we cannot handle it. First off, the Squid cache would need to contain one entry per language per page, rather than simply one entry per language. This means multiple entries for the same URL that are varied between based on Accept-Language (fragmentation), which in turn means the size of the Squid cache would explode: if there are, say, 20 popular languages out there that cause significant cache population (excluding English), the cache size for Commons would be roughly multiplied by 20, as would the number of render requests to the Apaches. Second, I believe that Squid currently doesn't even support this kind of fragmentation, but I may be wrong. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 03/12/2010 02:14 PM, Maarten Dammers wrote: Hi, Billinghurst schreef: In fact, it would be interesting to see some of the data of images on small and large wikis, including the numbers transferred from each to Commons, and who undertook such actions (local vs Commons users). I have some statistics at http://commons.wikimedia.org/wiki/User:Multichill/Commonscat_stats You can see that of the bigger Wikipedia's es (Spanish), pt (Portugese), sv (Swedish), nl (Dutch), cs (Czech), sk (Slovak) da (Danish) seem to have moved to Commons. Also pl (Polish) seems to be moving a lot of images to Commons yes, pl.wiki is going currently even further. There is a discussion to almost completely disable the possibility to upload files locally and reroute all uploads to Commons. masti ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Roan Kattouw schrieb: 2010/3/12 Marcus Buck w...@marcusbuck.org: Can you please elaborate? And feel free to use technical terms ;-) Why would that be a problem? We can cache the English pages so why can't we cache non-English pages? Of course the amount of rendering events will rise, but I cannot imagine why this rise would be so immense we cannot handle it. First off, the Squid cache would need to contain one entry per language per page, rather than simply one entry per language. This means multiple entries for the same URL that are varied between based on Accept-Language (fragmentation), which in turn means the size of the Squid cache would explode: if there are, say, 20 popular languages out there that cause significant cache population (excluding English), the cache size for Commons would be roughly multiplied by 20, as would the number of render requests to the Apaches. Second, I believe that Squid currently doesn't even support this kind of fragmentation, but I may be wrong. Perhaps I'm totally wrong, my knowledge of squid is somewhere between non-existant and sketchy, but my impression was that squid uses cache keys and that any information can be coded into these cache keys. (At least that's what I recall from the time we switched the local file description pages transcluded from Commons from English-only to the local projects language.) About the size explosion: do we hit any hard limitations if the squid cache multiplies its size? Like none of our servers has enough hard disk space to hold a commons squid cache 20 times the current size or what is it? By the way, what is the current cache size? Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 9:58 AM, Marcus Buck w...@marcusbuck.org wrote: Roan Kattouw schrieb: 2010/3/12 Marcus Buck w...@marcusbuck.org: Can you please elaborate? And feel free to use technical terms ;-) Why would that be a problem? We can cache the English pages so why can't we cache non-English pages? Of course the amount of rendering events will rise, but I cannot imagine why this rise would be so immense we cannot handle it. First off, the Squid cache would need to contain one entry per language per page, rather than simply one entry per language. This means multiple entries for the same URL that are varied between based on Accept-Language (fragmentation), which in turn means the size of the Squid cache would explode: if there are, say, 20 popular languages out there that cause significant cache population (excluding English), the cache size for Commons would be roughly multiplied by 20, as would the number of render requests to the Apaches. Second, I believe that Squid currently doesn't even support this kind of fragmentation, but I may be wrong. Perhaps I'm totally wrong, my knowledge of squid is somewhere between non-existant and sketchy, but my impression was that squid uses cache keys and that any information can be coded into these cache keys. (At least that's what I recall from the time we switched the local file description pages transcluded from Commons from English-only to the local projects language.) That uses Memcached, not Squid. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Chad schrieb: On Fri, Mar 12, 2010 at 9:58 AM, Marcus Buck w...@marcusbuck.org wrote: Roan Kattouw schrieb: 2010/3/12 Marcus Buck w...@marcusbuck.org: Can you please elaborate? And feel free to use technical terms ;-) Why would that be a problem? We can cache the English pages so why can't we cache non-English pages? Of course the amount of rendering events will rise, but I cannot imagine why this rise would be so immense we cannot handle it. First off, the Squid cache would need to contain one entry per language per page, rather than simply one entry per language. This means multiple entries for the same URL that are varied between based on Accept-Language (fragmentation), which in turn means the size of the Squid cache would explode: if there are, say, 20 popular languages out there that cause significant cache population (excluding English), the cache size for Commons would be roughly multiplied by 20, as would the number of render requests to the Apaches. Second, I believe that Squid currently doesn't even support this kind of fragmentation, but I may be wrong. Perhaps I'm totally wrong, my knowledge of squid is somewhere between non-existant and sketchy, but my impression was that squid uses cache keys and that any information can be coded into these cache keys. (At least that's what I recall from the time we switched the local file description pages transcluded from Commons from English-only to the local projects language.) That uses Memcached, not Squid. Okay, my fault. But according to http://www.mail-archive.com/squid-us...@squid-cache.org/msg01131.html squid supports it since version 2.5. Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
* Casey Brown li...@caseybrown.org [Fri, 12 Mar 2010 07:13:36 -0500]: On Fri, Mar 12, 2010 at 7:02 AM, Dmitriy Sintsov ques...@rambler.ru wrote: Hmm.. it seems that commons already uses uselang parameter. It just should use it persistantly and the parameter should be passed via the links from small wikis. You can change the language of the Commons interface in your preferences, instead of using the ?uselang= parameter all the time. As other peoples correctly pointed out, it is not convenient to select the language manually in user's options (especially for anonymous, who should not have options). One expects his smaller wiki language code to be seamlessly copied to commons (perhaps by passing uselang then making it persistent via the cookie). It can be done for anonymous, too. If that would hugely increase squid servers load, then perhaps only few selected namespaces (NS_SPECIAL's, Commons: whatever the code is) will use such cookie, at least. This page uses links This project page in other languages: at the top to swich uselang. It can be done automatically. http://commons.wikimedia.org/wiki/Commons:Upload Dmitriy ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 9:09 AM, Maarten Dammers maar...@mdammers.nl wrote: I understand this makes caching more difficult, but did anyone ever do measurements? Without decent metrics this is just wild guessing. Things to measure for example: * Total text requests for all sites * Total text requests for just Commons * Total misses for all sites * Total misses for just Commons I'm sure someone can dig up these metrics. All this stuff is measured and stored in various places, yes. I don't have the data handy, but the Squid hit rate is well over 95% last I heard for simple page views. Worst case all text requests for Commons go to the backend directly, right? Yes, and this is disastrous. It would mean load on the application servers would increase by a factor of twenty or a hundred or more. Even if only for Commons, it could be a huge increase in load. Keeping the Squid hit rate as high as possible is absolutely essential to keep the site running properly, barring a huge investment in new hardware. Remember that if you even create two variants that are widely used, you've come reasonably close to doubling load on the backend. Only one person from each variant needs to view any given revision of a page to force the backend to generate two copies instead of one. Causing visits to Commons to be split into twenty different languages might double backend load by itself. That's just not tenable. In principle, one could imagine hacking Squid to be smart enough to cache contents and interface separately, and paste them together on view about as quickly as it can serve plain requests now. But I don't know how feasible that would be in practice. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
* Aryeh Gregor simetrical+wikil...@gmail.com [Fri, 12 Mar 2010 12:31:39 -0500]: In principle, one could imagine hacking Squid to be smart enough to cache contents and interface separately, and paste them together on view about as quickly as it can serve plain requests now. But I don't know how feasible that would be in practice. Perhaps an uploading from small wiki to commons via api while small wiki localizes client output with it's own language messages. Or, perhaps some parts of localization can be done with Javascript (which can read cookies, too), again not altering squid caching. Dmitriy ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 12:44 PM, Dmitriy Sintsov ques...@rambler.ru wrote: Perhaps an uploading from small wiki to commons via api while small wiki localizes client output with it's own language messages. Or, perhaps some parts of localization can be done with Javascript (which can read cookies, too), again not altering squid caching. As long as we're only talking about uploads, the simplest solution is just to set the language for auto-created SUL accounts to the language on the home wiki instead of the local default language. Then all you need is good localization of Commons messages and you're pretty much set. Unregistered users can't upload to Commons, after all. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Aryeh Gregor hett schreven: Worst case all text requests for Commons go to the backend directly, right? Yes, and this is disastrous. It would mean load on the application servers would increase by a factor of twenty or a hundred or more. Even if only for Commons, it could be a huge increase in load. Keeping the Squid hit rate as high as possible is absolutely essential to keep the site running properly, barring a huge investment in new hardware. Remember that if you even create two variants that are widely used, you've come reasonably close to doubling load on the backend. Only one person from each variant needs to view any given revision of a page to force the backend to generate two copies instead of one. Causing visits to Commons to be split into twenty different languages might double backend load by itself. That's just not tenable. In principle, one could imagine hacking Squid to be smart enough to cache contents and interface separately, and paste them together on view about as quickly as it can serve plain requests now. But I don't know how feasible that would be in practice. Content on Commons is language-dependant too, so that's no solution. Can anybody please give some numbers? How severely would the squid hit rate go down? How many servers would be necessary to compensate? How much would this cost? We need numbers to make a decision like that. Please spend some seconds to think about what we are speaking: At the moment a user needs to be logged-in and additionally he needs to enter the preferences menu (which is in English) and select a language he knows. But most people do not change the preferences. Most people are not even logged in. Most people do not even know English (at least 3/4 of the world's population). We render Commons unusable or at least less usable for at least 90% of the world. Basically it's exactly the same situation as back in 2001 when people discussed whether Wikipedia should go multilingual. I hope nobody rejects that step even though it would have saved us so much resources in the last 9 years. Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 12:59 PM, Marcus Buck w...@marcusbuck.org wrote: Can anybody please give some numbers? How severely would the squid hit rate go down? How many servers would be necessary to compensate? How much would this cost? We need numbers to make a decision like that. I don't have the numbers. Try asking someone like Domas Mitzuas or Tim Starling. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/12 Marcus Buck w...@marcusbuck.org: Please spend some seconds to think about what we are speaking: At the moment a user needs to be logged-in and additionally he needs to enter the preferences menu (which is in English) and select a language he knows. But most people do not change the preferences. Most people are not even logged in. Most people do not even know English (at least 3/4 of the world's population). We render Commons unusable or at least less usable for at least 90% of the world. I realize this is serious, but it's somewhat mitigated by the fact that Squid is only a limiting factor for anonymous users: we can still honor Accept-Language for logged-in users without affecting anything on the Squid side. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Roan Kattouw hett schreven: 2010/3/12 Marcus Buck w...@marcusbuck.org: Please spend some seconds to think about what we are speaking: At the moment a user needs to be logged-in and additionally he needs to enter the preferences menu (which is in English) and select a language he knows. But most people do not change the preferences. Most people are not even logged in. Most people do not even know English (at least 3/4 of the world's population). We render Commons unusable or at least less usable for at least 90% of the world. I realize this is serious, but it's somewhat mitigated by the fact that Squid is only a limiting factor for anonymous users: we can still honor Accept-Language for logged-in users without affecting anything on the Squid side. I don't think we need accept-language for logged-in users. We have preference settings for logged-in users. And if we want to improve the situation of logged-in users we should go with the idea Maarten Dammers and I proposed: Make the language set in the preferences of the home wiki the default for users who are new on Commons. I don't know how many of Commons' visitors are anonymous. But either it is many, then it is important to support them better, or it is few, then the impact cannot be great. We need some numbers! Marcus Buck ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
In reply to: Message: 9 Date: Fri, 12 Mar 2010 20:44:33 +0300 From: Dmitriy Sintsov ques...@rambler.ru Subject: Re: [Wikitech-l] Uploads on small wikis To: Wikimedia developers wikitech-l@lists.wikimedia.org Message-ID: 596823262.1268415874.80897352.68...@mcgi68.rambler.ru Content-Type: text/plain; charset=us-ascii; format=flowed * Aryeh Gregor simetrical+wikil...@gmail.com [Fri, 12 Mar 2010 12:31:39 -0500]: In principle, one could imagine hacking Squid to be smart enough to cache contents and interface separately, and paste them together on view about as quickly as it can serve plain requests now. But I don't know how feasible that would be in practice. Perhaps an uploading from small wiki to commons via api while small wiki localizes client output with it's own language messages. Or, perhaps some parts of localization can be done with Javascript (which can read cookies, too), again not altering squid caching. Dmitriy Stupid question, if having the ?uselang parameter can be done without significant problems, wouldn't having javascript that just appeneds ?uselang= whatevever its set to to all internal links on a page whenever the url for the current page has ?uselang=something also not cause significant problems (well making uselang persist)? (although then again, I'd imagine that would not be much different then the use a cookie solution in terms of caching) -bawolff ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 4:48 PM, bawolff bawolff...@gmail.com wrote: Stupid question, if having the ?uselang parameter can be done without significant problems, wouldn't having javascript that just appeneds ?uselang= whatevever its set to to all internal links on a page whenever the url for the current page has ?uselang=something also not cause significant problems (well making uselang persist)? The reason uselang doesn't cause significant problems is because it's not used very heavily. If it persisted across page views, a lot more people would probably use it. This is really a case where performance and functionality are at odds -- but I'm not knowledgeable enough to say what the right tradeoff is here. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On Fri, Mar 12, 2010 at 8:44 AM, Roan Kattouw roan.katt...@gmail.com wrote: That would severely break the Squid cache. In non-technical terms: if we enable this and people actually use it, the site goes down. I've wanted a way to preserve the language for anoymous users for years. (On foundationwiki in particular, since there's no way for people to login and set their preferences language on that wiki.) Translatewiki uses [[Extension:LanguageSelector]], which instead sets a cookie for anonymous users (we'd use ?setlang instead of ?uselang)... but that doesn't work with the squid configuration. :-( On Fri, Mar 12, 2010 at 7:02 AM, Dmitriy Sintsov ques...@rambler.ru wrote: Hmm.. it seems that commons already uses uselang parameter. It just should use it persistantly and the parameter should be passed via the links from small wikis. There's a bug about this, by the way: https://bugzilla.wikimedia.org/show_bug.cgi?id=4125 -- Casey Brown Cbrown1023 ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Dmitriy Sintsov wrote: Casey Brown: You can change the language of the Commons interface in your preferences, instead of using the ?uselang= parameter all the time. As other peoples correctly pointed out, it is not convenient to select the language manually in user's options (especially for anonymous, who should not have options). One expects his smaller wiki language code to be seamlessly copied to commons (perhaps by passing uselang then making it persistent via the cookie). It can be done for anonymous, too. SUL accounts should have a list of languages the user can (is willing to) read. When he visits a wiki, if the wiki's preferred language matches one of those, use it. Else use the most similar language to the wiki preferred language from those that the user has. Some users will probably still want an override, though. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Uploads on small wikis
Hello all, together with Frank Schulenburg and Naoko Komura, I just participated in a video-conference with the winners of the Google Kiswahili Wikipedia challenge ( http://www.google.com/events/kiswahili-wiki/ ), and we talked about some of the challenges they encountered when contributing to the Swahili Wikipedia. One of the issues was that it was very hard for them to upload files. Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. From a user experience standpoint, this is horrible. For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Thanks, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/11 Huib Laurens sterke...@gmail.com: When that restriction is lifted it would be much more easy to upload copyvios, and maybe the smaller communities can't really handle it. I'm sure that's true, but I think that's OK. Small wikis need some time to grow and sort these things out on their own; trying to solve this problem by making it impossible to upload for new users is IMO the wrong approach. -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 03/11/2010 07:06 PM, Erik Moeller wrote: For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Instead, why not redirect people with no permissions straight to Commons? I think you can upload there when not auto-confirmed. Some projects (en.wiktionary for example) already redirect everyone to commons when they click upload. Yours Conrad Thanks, Erik ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
2010/3/11 Conrad Irwin conrad.ir...@googlemail.com Instead, why not redirect people with no permissions straight to Commons? I think you can upload there when not auto-confirmed. Some projects (en.wiktionary for example) already redirect everyone to commons when they click upload. Yours Conrad On Commons you can upload right away, but Commons doesn't have a good support for small languages that can scare people away :( Huib -- The soldiers graves are great preachers of peace http://commons.wikimedia.org/wiki/User:Abigor ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
On 3/11/2010 2:06 PM, Erik Moeller wrote: Hello all, together with Frank Schulenburg and Naoko Komura, I just participated in a video-conference with the winners of the Google Kiswahili Wikipedia challenge ( http://www.google.com/events/kiswahili-wiki/ ), and we talked about some of the challenges they encountered when contributing to the Swahili Wikipedia. One of the issues was that it was very hard for them to upload files. Specifically, when you're a new user on a small wiki like sw.wp, you _cannot upload_ locally due to a restriction of uploads to autoconfirmed users. The upload link isn't even visible in the sidebar until you're autoconfirmed, and you get a confusing error message if you happen to call up Special:Upload. From a user experience standpoint, this is horrible. For the immediate future, I suggest lifting this restriction for wikis between 1,000 and 50,000 articles in size (large enough to have a few active users, small enough to not yet have lots of policy around these issues). Ultimately we'll want to integrate Commons better into the user experience, but until then, IMO we should eliminate artificial impediments like this which prevent people from growing their communities and frustrate them -- unless there's a proven issue of large scale abuse. Does that make sense? Thanks, Erik I do hope the concerns that led to this being implemented[1] will be taken into consideration before it is undone. [1] http://meta.wikimedia.org/w/index.php?title=Wikimedia_Forumoldid=931972#Set_upload_to_autoconfirmed_Wikimedia-wide -- Alex (wikipedia:en:User:Mr.Z-man) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Uploads on small wikis
Hoi, I am really happy that we want to help the smaller Wikipedias with uploading their pictures. It helps when it is made easy. One of the ways in which we can make it easy is to reduce the number of choices involved. When the only choice is cc-by-sa, we only need to explain one license. When the software involved is properly localised in their language, confusion will be a lot less. One of the ways that makes a big difference to people is, when they know that a small number of messages provide exactl functionality.. The WikiReader and the mobile Wikimedia experience point in that direction. There are however two observations; it produces overhead at translatewiki.net and it helps when you target people who are actively translating for that language. What you can do is make the local upload facility available once these limited number of messages have been localised. In this way it is a reward and not depended on an arbitrary number. When these messages are the same messages as used for Commons itself ... When there are clear differences in procedure local vis a vis Commons, it makes sense to have a document in English that explains these differences. To make it less problematic to move material to Commons, it might be an idea when they can ask Commons admins / volunteers to audit material and mark material that are Commons compliant. These files can then be moved to Commons. When regularly a percentage is calculated of the material that is OK for Commons versus the total amount of messages, we provide at least a clue what will happen when they grow to big to have too much illegal material. Thanks, GerardM On 11 March 2010 20:14, Erik Moeller e...@wikimedia.org wrote: 2010/3/11 Huib Laurens sterke...@gmail.com: When that restriction is lifted it would be much more easy to upload copyvios, and maybe the smaller communities can't really handle it. I'm sure that's true, but I think that's OK. Small wikis need some time to grow and sort these things out on their own; trying to solve this problem by making it impossible to upload for new users is IMO the wrong approach. -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
