Re: [Wikitech-l] [Foundation-l] building lucene-search-2 binary?

[Anon Sricharoenchai]

On 8/29/09, Robert Stojnic  wrote:
>
>  The foundation-l is the wrong mailing list, please use the talk page for
> lucene-search on mediawiki.org.

Sorry, it's my mistake.
I intend to post in wikitech-l.

>
>  r.
>
>  Anon Sricharoenchai wrote:
>
> > Hi,
> >
> > Some question about lucene-search-2 binary,
> >
> > 1. Do I need apache's lucene library when building lucene-search-2 binary?
> >Or it is already included in lucene-search-2 source tree?
> >
> >
>
>  already there
>
>
> > 2. Did lucene-search-2 in wikimedia (before migration to 2.1) use the
> > same binary as found in sourceforge,
> >http://sourceforge.net/projects/lucene-search/files/ ?
> >
> >
>
>  no, we always use the latest svn
>
>
> > 3. Which version of lucene library is used to build binary found in
> > sourceforge and in wikimedia?
> >
> >
>
>  i think it is 2.3, but it is in lib/
>
>
> > Best regards,
> > Anon.
> >
> > ___
> > foundation-l mailing list
> > foundatio...@lists.wikimedia.org
> > Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> >
> >
>
>

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Bugzilla Weekly Report

[reporter]

MediaWiki Bugzilla Report for August 24, 2009 - August 31, 2009

Status changes this week

Bugs NEW   :  115 
Bugs ASSIGNED  :  7   
Bugs REOPENED  :  13  
Bugs RESOLVED  :  78  

Total bugs still open: 3843

Resolutions for the week:

Bugs marked FIXED  :  39  
Bugs marked REMIND :  0   
Bugs marked INVALID:  11  
Bugs marked DUPLICATE  :  20  
Bugs marked WONTFIX:  6   
Bugs marked WORKSFORME :  3   
Bugs marked LATER  :  0   
Bugs marked MOVED  :  0   

Specific Product/Component Resolutions & User Metrics 

New Bugs Per Component

Site requests   6   
General/Unknown 5   
AbuseFilter 4   
LiquidThreads   4   
Semantic MediaWiki  3   

New Bugs Per Product

MediaWiki   16  
Wikimedia   11  
MediaWiki extensions21  

Top 5 Bug Resolvers

alex.emsenhuber [AT] bluewin.ch 33  
innocentkiller [AT] gmail.com   13  
roan.kattouw [AT] gmail.com 4   
hcatlin [AT] wikimedia.org  3   
Bryan.TongMinh [AT] Gmail.com   3   


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Aryeh Gregor]

On Sun, Aug 30, 2009 at 8:37 PM, Platonides wrote:
> You get a request for page Foo:Bar. It could be a page name, or Foo
> could mean Special in Bantu. How do you check (efficiently) over 300
> languages?

Ouch.  Okay, that's out for namespaces.  It should work fine for
special page names, though, right?  Although I guess that's
semi-pointless if the "Special:" part doesn't work.

Clearly it was a bad idea to use a character for namespace separator
that's allowed in page names.  Crazy idea, but maybe we could still
change . . . would it cause conflicts to use ">", say?  Then we could
display the namespace in a breadcrumb-y fashion, with spaces on either
side.  Obviously we'd accept ":" forever as well, for compatibility.
I'd think it would be pretty easy to get most stuff working with a new
namespace separator, just some hackery with Title::secureAndSplit()
should do it.

> I'd prefer an option (stored as a global preference) to use canonical
> names instead of localised ones. It doesn't need to redirect from
> localised to canonical, simply keep the url which was inputted.
>
> It's uncomfortable browsing to xy.wikipedia.org/wiki/Special:something,
> then moving to check it on yx.wikpedia, and having to go back to retype
> the pagename because "Spécialis:somethingelese doesn't exist".
> Having canonical pagenames also used to be helpful when browsing a wiki
> on a foreign language to determine which link lead to eg. the
> contributions of a user, by hovering the different options (now you will
> need to change to ?uselang=).

This should work fine with a global language preference, shouldn't it?

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Platonides]

Aryeh Gregor wrote:
> Tangentially, is there any actual reason we couldn't allow any
> language's alias (e.g., for special page names or namespaces) to work
> in any other language?  We'd have to make sure we have a moderately
> sane way of resolving conflicts, but those should be extremely
> uncommon.

You get a request for page Foo:Bar. It could be a page name, or Foo
could mean Special in Bantu. How do you check (efficiently) over 300
languages?
I'd prefer an option (stored as a global preference) to use canonical
names instead of localised ones. It doesn't need to redirect from
localised to canonical, simply keep the url which was inputted.

It's uncomfortable browsing to xy.wikipedia.org/wiki/Special:something,
then moving to check it on yx.wikpedia, and having to go back to retype
the pagename because "Spécialis:somethingelese doesn't exist".
Having canonical pagenames also used to be helpful when browsing a wiki
on a foreign language to determine which link lead to eg. the
contributions of a user, by hovering the different options (now you will
need to change to ?uselang=).
That's a feature that wouldn't be used by most users, but people doing
crosswiki work, like stewards or members of SWMT, would surely find it
useful.


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Aryeh Gregor]

On Sun, Aug 30, 2009 at 5:24 PM, Marcus Buck wrote:
> But that would be inconsistent with us localising special page names
> which too appear in URLs only.

Special page names also appear in the title of the page when you view
it, which is a very visible location.  And on Special:AllPages, and
probably other places.  It's not the same at all.

On Sun, Aug 30, 2009 at 7:27 PM, Brion Vibber wrote:
> They're *certainly* comparable -- namespaces and special page names have
> canonical names (ASCII-friendly English), per-language localizations and
> aliases, and locally-configured aliases and overrides (via config file).

Well, yes, but we could add those for anything.  We *could* add
per-language localizations for GET parameters too, if we liked; it
wouldn't be too hard, just add some magic to WebRequest and start
converting HTML outputters.  But that would be unreasonable.  I agree
that more localization is good, but where's the dividing line?  The
usergroup name in this case is only a GET parameter value dressed up
as a subpage.  Should we also localize log subpages?  How many other
special pages behave like this?

> This is a trade-off which is usually friendlier to most users (they can
> read the text in URLs in their own language) while moderately annoying
> to some power users use cases (copy-pasting a path from one site to
> another may not work, it may be harder to identify pages on a site whose
> language you do not speak but are performing maintenance editing or
> debugging on).

Tangentially, is there any actual reason we couldn't allow any
language's alias (e.g., for special page names or namespaces) to work
in any other language?  We'd have to make sure we have a moderately
sane way of resolving conflicts, but those should be extremely
uncommon.

> It would not be terribly difficult to use content-language localized
> forms on the special page URLs where we most prominently use group
> names, or at least to accept them if provided; the primary problem area
> I see is in providing backwards-compatibility when the localizations change.

For Special:ListUsers, a possibly more serious problem is that the
localized group name might conflict with a username, since the
parameter is overloaded.  We avoid conflicts right now only because
canonical group names are conventionally all lowercase, and usernames
can't start with a lowercase letter.  (At least not a lowercase ASCII
letter?)

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Brion Vibber]

On 8/30/09 8:08 PM, Happy-melon wrote:
> "Marcus Buck"  wrote in message
> news:4a9aee20.7000...@marcusbuck.org...
>> But that would be inconsistent with us localising special page names
>> which too appear in URLs only. So in my opinion they should be localized.
>
> Not true: special page names are localised in the JS variables (wgPageName,
> for instance), and in the class applied to; and of course in links
> everywhere across the site.  The two aren't at all comparable.  The
> comparison of user or page ids is much more accurate.

They're *certainly* comparable -- namespaces and special page names have 
canonical names (ASCII-friendly English), per-language localizations and 
aliases, and locally-configured aliases and overrides (via config file).

We've always exposed the content-language localized variants of the 
namespaces to URLs, and started doing this for special page names as 
well in more recent years.

This is a trade-off which is usually friendlier to most users (they can 
read the text in URLs in their own language) while moderately annoying 
to some power users use cases (copy-pasting a path from one site to 
another may not work, it may be harder to identify pages on a site whose 
language you do not speak but are performing maintenance editing or 
debugging on).


User groups have canonical names (usually ASCII-friendly English), and 
two localized forms (for individual members and for the group as a 
whole), which are set via the localization system or locally configured 
via MediaWiki: messages.

It would not be terribly difficult to use content-language localized 
forms on the special page URLs where we most prominently use group 
names, or at least to accept them if provided; the primary problem area 
I see is in providing backwards-compatibility when the localizations change.

With namespace names and special page names maintained in the 
localization files or via site configuration, we take great care to keep 
old names as aliases when they're changed in order to keep old links 
working if possible.

There's currently no similar alias infrastructure for localized group 
names, which are provided and formatted for use in UI output and not 
(yet?) intended for URL input.

-- brion

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Happy-melon]

"Marcus Buck"  wrote in message 
news:4a9aee20.7000...@marcusbuck.org...
> But that would be inconsistent with us localising special page names
> which too appear in URLs only. So in my opinion they should be localized.

Not true: special page names are localised in the JS variables (wgPageName, 
for instance), and in the class applied to ; and of course in links 
everywhere across the site.  The two aren't at all comparable.  The 
comparison of user or page ids is much more accurate.

--HM 



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Marcus Buck]

Aryeh Gregor hett schreven:
> The parameter in the URL currently must be an exact match for the name
> of the group.  Group names are normally short, lowercase, ASCII,
> alphabetic English names that are consistent across wikis.  You can
> think of them like numbers, as if the URL were
> /w/index.php?title=Special:ListUsers&groupid=14 or something; they're
> meant to be opaque identifiers for use by system administrators, not
> user-visible strings.
>
> Do we expose group names anywhere other than URLs and message names?
> Message names aren't really an issue, since those are entirely English
> anyway.  I think it's not worth the effort of localizing if they only
> appear in URLs.  We already have some English in the
> http://pt.wikipedia.org/wiki/ part of the URL anyway, and English in
> all the GET parameter names, and so on.
But that would be inconsistent with us localising special page names 
which too appear in URLs only. So in my opinion they should be localized.

Marcus Buck
User:Slomox

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Aryeh Gregor]

On Sun, Aug 30, 2009 at 1:58 PM, Helder Geovane Gomes de
Lima wrote:
> Does anybody knows why the link
> http://en.wikipedia.org/wiki/Special:ListUsers/autoreviewer
> makes the target page to have the box on the right filled with
> "Autoreviewers" and shows only the users in that group, while using a
> translated name at pt.wikipedia, like
> http://pt.wikipedia.org/wiki/Especial:Lista_de_utilizadores/Robôs
> doesn't do the same thing? (in this example, we get a list of all users,
> starting from "Robôs")
>
> Is it possible to use a translated name in the link for this kind of special
> page? (lists of users in a group)

The parameter in the URL currently must be an exact match for the name
of the group.  Group names are normally short, lowercase, ASCII,
alphabetic English names that are consistent across wikis.  You can
think of them like numbers, as if the URL were
/w/index.php?title=Special:ListUsers&groupid=14 or something; they're
meant to be opaque identifiers for use by system administrators, not
user-visible strings.

Do we expose group names anywhere other than URLs and message names?
Message names aren't really an issue, since those are entirely English
anyway.  I think it's not worth the effort of localizing if they only
appear in URLs.  We already have some English in the
http://pt.wikipedia.org/wiki/ part of the URL anyway, and English in
all the GET parameter names, and so on.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Links to Special:ListUsers/...

[Helder Geovane Gomes de Lima]

Hello!

Does anybody knows why the link
http://en.wikipedia.org/wiki/Special:ListUsers/autoreviewer
makes the target page to have the box on the right filled with
"Autoreviewers" and shows only the users in that group, while using a
translated name at pt.wikipedia, like
http://pt.wikipedia.org/wiki/Especial:Lista_de_utilizadores/Robôs
doesn't do the same thing? (in this example, we get a list of all users,
starting from "Robôs")

Is it possible to use a translated name in the link for this kind of special
page? (lists of users in a group)

Helder
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Links to Special:ListUsers/...

[Thomas Dalton]

2009/8/30 Helder Geovane Gomes de Lima :
> Hello!
>
> Does anybody knows why the link
> http://en.wikipedia.org/wiki/Special:ListUsers/autoreviewer
> makes the target page to have the box on the right filled with
> "Autoreviewers" and shows only the users in that group, while using a
> translated name at pt.wikipedia, like
> http://pt.wikipedia.org/wiki/Especial:Lista_de_utilizadores/Robôs
> doesn't do the same thing? (in this example, we get a list of all users,
> starting from "Robôs")
>
> Is it possible to use a translated name in the link for this kind of special
> page? (lists of users in a group)

It seems you have to use the English name, which is presumably the
name used in the code. It is probably possible to make it so aliases
work - I suggest leaving a feature request on
https://bugzilla.wikimedia.org/ .

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Wikipedia iPhone app official page?

[Aryeh Gregor]

On Sat, Aug 29, 2009 at 8:37 AM, Brion Vibber wrote:
> One of my main concerns with a git transition though is figuring out how
> to divide up the repository into manageable pieces; our SVN repo
> includes many different projects including MediaWiki core, lots of
> extensions, dump processing tools, our custom Ubuntu packages for
> Wikimedia server deployment, our load balancing tools, etc.

I'd say put extensions and core in one repo, and make different repos
for everything else that's logically separate and unlikely to move
from repo to repo.  Or we could have core in one repo, and use
submodules for extensions.

On Sat, Aug 29, 2009 at 9:07 AM, Dmitriy Sintsov wrote:
> Some local coder told me that GIT is slower and consumes much more RAM
> on some operations than SVN.
> I can't confirm that, though, because I never used GIT and still rarely
> use SVN. But, be warned.

I hope we can do better than vague second-hand rumors when considering
easily quantifiable things like performance.  In my experience, git is
much faster than SVN on many operations simply because it doesn't need
to go to the server.  blame is often fast enough that you can do them
without feeling much urge to switch to another window while you wait.
log, diff (even for old revisions), etc. are nearly instantaneous.

Just out of interest, I once timed a fresh SVN checkout of trunk/ from
svn.wikimedia.org to the toolserver (so they were even in the same
datacenter).  Then I tested a git clone of a git-svn checkout of the
*entire* mediawiki/ repository, including trunk, branches, tags, and
all history, from a server I have (in Denver) to the toolserver (in
Amsterdam).  The git clone was *significantly* faster.  (To be fair,
it *was* http:// vs. git://, but still.)

Another interesting data point: an SVN checkout of trunk/, with
working copy, is 696M.  A git svn checkout of the entire repository
with branches, tags, and all history, plus working copy, is 661M after
git gc.  (But admittedly, before git gc it was 1005M.  Not too bad
anyway: 44% more for branches+tags+history.)

So I'm going to say that this claim completely contradicts my
experience using both git and SVN (and I've used both fairly
extensively).  Of course, I'm sure there are "some operations" where
git is slower than SVN, but that's not a very useful claim, given that
the converse is certainly true.

The only time I've found where git performance was totally
unacceptable when I tried to use it for compression/versioning of >1G
database backups -- git add used up multiple gigabytes of RAM and
OOMed when I tried to add the first file.  The lesson there,
obviously, is that git is meant to version source code, not files as
large as database backups.  git is known not to do so well on
extremely large repos.  It also has some trouble if you have lots of
large binary files that change a lot -- those can't be compressed
easily, especially if they're already compressed somehow.

On Sat, Aug 29, 2009 at 5:38 PM, Daniel
Friesen wrote:
> As a thought we could probably structure it something like this:
> - We have a phase3 repo for MediaWiki itself

One objection: could we please rename phase3 something sensible like
"mediawiki" if we do this?  :)

On Sat, Aug 29, 2009 at 6:48 PM, Marco
Schuster wrote:
> And so to the disk. If the disk or the controller sucks or is simply old
> (not everyone has shiny new hardware), you're also damn slow.

Um, and SVN won't be?  Do you have actual benchmarks indicating that
git is slower than svn on *any* typical distributed source-control
workload?

(By "distributed" I mean "with the remote server over the Internet".
I could well believe that if you're in an office where the remote
server is on a LAN, and the remote server happens to have 8 15k RPM
SCSIs and 16G RAM while your workstation is a three-year-old consumer
desktop, SVN would be a lot faster.  SVN might be better for other
things as well, like anything involving very large files or very large
repos.  But none of these are applicable to us.)

> What should
> also not be underestimated is the diskspace demand of a GIT repo

Have you actually tested the disk space usage side-by-side?  Because
my figures (from above) indicate that git doesn't use much more disk
space than SVN at all.  SVN doesn't compress its .svn directories at
all AFAICT -- git uses extremely heavy-duty compression.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SVN commit access

[Jan Luca]

Hello,

you must use your private key and not a "password".

For Linux: Run ssh-add /path/to/your/private/key/file
For Windows: Read
http://wiki.apisnetworks.com/index.php/Subversion#Windows_Setup

Viele Grüße
Jan

-Ursprüngliche Nachricht-
Von: wikitech-l-boun...@lists.wikimedia.org
[mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von Dmitriy
Sintsov
Gesendet: Freitag, 28. August 2009 10:51
An: Wikimedia developers
Betreff: Re: [Wikitech-l] SVN commit access

* Bryan Tong Minh  [Fri, 28 Aug 2009 10:33:54 
+0200]:
> A checkout on that url should work anonymously. Anyway, a read-write
Sorry, I've copypasted the wrong url.

> checkout should be done from
> svn+ssh://svn.wikimedia.org/svnroot/mediawiki/trunk/phase3
>
Of course, this one doesn't work:

svn checkout 
svn+ssh://ques...@svn.wikimedia.org/svnroot/mediawiki/trunk/phase3/extension
s 
wiki/extensions
ques...@svn.wikimedia.org's password:
Permission denied, please try again.
ques...@svn.wikimedia.org's password:

Dmitriy

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Have wikimedia lucene-search config been published?

[K. Peachey]

On Sat, Aug 29, 2009 at 12:14 AM, Robert Stojnic wrote:
>
> We could publish lsearch-global.conf since it doesn't contain any
> private info, and there is only one. However, lsearch.conf is different
> for various searchers and contains passwords to access the OAI feed.
>
> r.
Could we perhaps strip the passwords and other sensitive info out of
it then publish it onto the NOC?.

-Peachey

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l