Re: [Wikitech-l] [Wikimania-l] Wikimania DevCamp

2013-07-30 Thread aude 
On Tue, Jul 30, 2013 at 8:02 AM, Lodewijk wrote:

>
> 2013/7/30 Deryck Chan 
>
>> Actually you do. Registering for Wikimania will be highly appreciated.
>>
>
> OK, but to clarify: if you are already registered for Wikimania (as most
> people who are on this mailing list and will be in Hong Kong) there is
> nothing else you'd have to do?
>

Exactly!  There is no additional registration process for DevCamp

Cheers,
Katie



>
> Thanks,
> Lodewijk
>
> ___
> Wikimania-l mailing list
> wikimani...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikimania-l
>
>


-- 
@wikimediadc / @wikidata
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] MediaWiki-Vagrant: new features

2013-07-30 Thread Antoine Musso 
Le 30/07/13 06:39, Ori Livneh a écrit :
> There's a short, 1-minute screencast up at
> http://ascii.io/a/4428demonstrating usage. Check it out. I'll wait.

I am sorry to report you are definitely too awesome.  Thank you for all
your work on that front \O/

-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Replication to the toolserver stopped

2013-07-30 Thread Silke Meyer 
Hi wikitech-l!

The db replication of s5 and s6 stopped on the toolserver. Merlissimo
searched for information and found that you stopped some of your
slaves that toolserver is using as master.
Is there an ETA when they will be back? Please provide some information!

Thanks und cheers, Silke

-- 
Silke Meyer
Internes IT-Management und Projektmanagement Toolserver

Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. (030) 219 158 260

http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Replication to the toolserver stopped

2013-07-30 Thread Antoine Musso 
Le 30/07/13 12:25, Silke Meyer a écrit :
> Hi wikitech-l!
> 
> The db replication of s5 and s6 stopped on the toolserver. Merlissimo
> searched for information and found that you stopped some of your
> slaves that toolserver is using as master.
> Is there an ETA when they will be back? Please provide some information!
> 
> Thanks und cheers, Silke

The very first step would be to fill in a bug against "Wikimedia Labs" >
"Infrastructure".   That will kick in mails to the person able to fix
the issue :-]

Also you really want to use the labs-l list, I am not sure any ops is
reading wikitech-l which is mostly for development/MediaWiki :-]


-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Replication to the toolserver stopped

2013-07-30 Thread Petr Bena 
She is talking about toolserver, not labs ;) I think this list is kind
of correct place

On Tue, Jul 30, 2013 at 3:59 PM, Antoine Musso  wrote:
> Le 30/07/13 12:25, Silke Meyer a écrit :
>> Hi wikitech-l!
>>
>> The db replication of s5 and s6 stopped on the toolserver. Merlissimo
>> searched for information and found that you stopped some of your
>> slaves that toolserver is using as master.
>> Is there an ETA when they will be back? Please provide some information!
>>
>> Thanks und cheers, Silke
>
> The very first step would be to fill in a bug against "Wikimedia Labs" >
> "Infrastructure".   That will kick in mails to the person able to fix
> the issue :-]
>
> Also you really want to use the labs-l list, I am not sure any ops is
> reading wikitech-l which is mostly for development/MediaWiki :-]
>
>
> --
> Antoine "hashar" Musso
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Voting disabled in bugzilla for some products

2013-07-30 Thread Andre Klapper 
On Mon, 2013-07-29 at 10:38 +0200, Lydia Pintscher wrote:
> I'd like to keep them at least for the Wikidata-related bugs. The
> votes there are taken into account.

That is great to hear and good to know!

So I assume that you go to
https://bugzilla.wikimedia.org/query.cgi?query_format=advanced , select
the components that you maintain and only include tickets that are in
open state, retrieve the search results (list of open tickets), and sort
the results by the "Votes" column?

For others on this list that would also like to see the Votes displayed
for tickets in the search results: 
Click "Change Columns" at the bottom of buglist.cgi and move "Votes"
from "Available Columns" to "Selected Columns".

Note: Changing the columns in search results is also explained in
http://blogs.gnome.org/aklapper/2013/06/21/bugzillatips-changecolumns/


Technical note: Voting in Bugzilla can be enabled on a product level
only, not per component (as the Wikidata components are under the
"MediaWiki extensions" product in Bugzilla).

andre
-- 
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wiktionary-l] Listing missing words of wiktionnaries

2013-07-30 Thread Mathieu Stumpf 

Le 2013-07-26 20:26, Amgine a écrit :
The request is to create a web-based text corpus[1] from which to 
derive

frequencies and then compare with existing wiktionaries. Not a light
undertaking, but one which has been proposed and implemented 
previously

(e.g. Connel's Gutenberg project[2])

Generically speaking, someone would need to determine the appropriate
size of the corpus sample, it's temporal currency, and the method of
creating and maintaining it. This isn't easy to do, and having no
strictures results in unwieldy and mostly irrelevant products like
Google's n-grams[3] (on the other hand, if someone can figure out how 
to
filter n-grams usefully it would mean we don't have to build our 
own.)


Actually, I think it would be interesting to have a trend history of 
words usage over centuries (current trend would also be interesting but 
probably harder to implement). Wikisource may be used in order to 
achieve that.




Amgine

[1] https://en.wikipedia.org/wiki/Linguistic_corpus
[2] https://en.wiktionary.org/wiki/User:Connel_MacKenzie/Gutenberg
[3] http://storage.googleapis.com/books/ngrams/books/datasetsv2.html


On 26/07/13 09:18, Lars Aronsson wrote:

On 07/23/2013 11:23 AM, Mathieu Stumpf wrote:
Here is what I would like to do : generating reports which give, 
for

a given language, a list of words which are used on the web with a
number evaluating its occurencies, but which are not in a given
wiktionary.

How would you recommand to implemente that within the wikimedia
infrastructure?


Some years back, I undertook to add entries for
Swedish words in the English Wiktionary. You can
follow my diary at http://en.wiktionary.org/wiki/User:LA2

Among the things I did was to extract a list of all
Swedish words that already had entries. The best
way was to use CatScan to list entries in categories
for Swedish words. Even if there is a page called
"men", this doesn't mean the Swedish word "men"
has an entry, because it could be the English word
"men" that is in that page.

Then I extracted all words from some known texts,
e.g. novels, the Bible, government reports, and the
Swedish Wikipedia, counting the number of
occurrencies of each word. Case significance is
a bit tricky. There should not be an entry for
lower-case stockholm, so you can't just convert
everything to lower case. But if a sentence begins
with a capital letter, that word should not have
a capitalized entry. Another tricky issue is
abbreviations, which should keep the period,
for example "i.e." rather than "i" and "e". But
the period that ends a sentence should be removed.
When splitting a text into words, I decided to keep
all periods and initial capital letters, even if this
leads to some false words.

When you have word frequency statistics for a text,
and a list of existing entries from Wiktionary, you
can compute the coverage, and I wrote a little
script for this. I found that English Wiktionary already
had Swedish entries covering 72% of the words in the
Bible, and when I started to add entries for the most
common of the missing words, I was able to increase
this to 87% in just a single month (September 2010).

Many of the common words that were missing when
I started were adverbs such as "thereof", "herein",
which occur frequently in any text but are not very
exciting to write entries about. This statistics-based
approach gave me a reason to add those entries.

It is interesting to contrast a given text to a given
dictionary in this way. The Swedish entries in the
English Wiktionary is a different dictionary than the
Swedish entries in the German or Danish Wiktionary.
The kinds of words found in the Bible are different
from those found in Wikipedia or in legal texts.
There is not a single, universal text corpus that we
can aim to cover. Google has released its ngram
dataset. I'm not sure if it covers Swedish, but even
if it does, it must differ from the corpus frequencies
published by the Swedish Academy.

It is relatively easy to extract a list of existing entries
from Wiktionary. But to prepare a given text corpus
for frequency and coverage analysis needs more
preparation.



___
Wiktionary-l mailing list
wiktionar...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiktionary-l


--
Association Culture-Libre
http://www.culture-libre.org/

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] MediaWiki-Vagrant: new features

2013-07-30 Thread Yuri Astrakhan 
Awesome!!! Thanks Ori!!!

Can we have varnish through a role please? We will have to do a lot of dev
& test for it soon.

The default varnish config in   /etc/varnish/default.vcl could have this at
the very top of the file. Leave all comments there.

backend default {
.host = "127.0.0.1";
.port = "80";
}

start with:

sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a
0.0.0.0:8080

This way varnish will only work if accessed through port 8080 (needs to be
mapped to outside), while port 80 will continue be served uncached. The
admin port 2000 doesn't have to be patched to the outside (rarely used, can
be accessed through ssh).

P.S. Note that varnish config should not be deleted on every vagrant up -
there could be a lot of changes during development to that file. Maybe it
would make sense to have an include command of some sort the same way as
settings.d/ files for vagrant.

Thanks!




On Tue, Jul 30, 2013 at 4:58 AM, Antoine Musso  wrote:

> Le 30/07/13 06:39, Ori Livneh a écrit :
> > There's a short, 1-minute screencast up at
> > http://ascii.io/a/4428demonstrating usage. Check it out. I'll wait.
>
> I am sorry to report you are definitely too awesome.  Thank you for all
> your work on that front \O/
>
> --
> Antoine "hashar" Musso
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] MediaWiki-Vagrant: new features

2013-07-30 Thread Tyler Romeo 
Ori I have a quick question.

Would there be any way to have a quick method of switching PHP versions,
specifically for testing purposes? Like "vagrant enable-role php5.4" or
maybe it'd have it's own command.

That way it's easy to test compatibility with other versions.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com


On Tue, Jul 30, 2013 at 12:51 PM, Yuri Astrakhan
wrote:

> Awesome!!! Thanks Ori!!!
>
> Can we have varnish through a role please? We will have to do a lot of dev
> & test for it soon.
>
> The default varnish config in   /etc/varnish/default.vcl could have this at
> the very top of the file. Leave all comments there.
>
> backend default {
> .host = "127.0.0.1";
> .port = "80";
> }
>
> start with:
>
> sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000-a
> 0.0.0.0:8080
>
> This way varnish will only work if accessed through port 8080 (needs to be
> mapped to outside), while port 80 will continue be served uncached. The
> admin port 2000 doesn't have to be patched to the outside (rarely used, can
> be accessed through ssh).
>
> P.S. Note that varnish config should not be deleted on every vagrant up -
> there could be a lot of changes during development to that file. Maybe it
> would make sense to have an include command of some sort the same way as
> settings.d/ files for vagrant.
>
> Thanks!
>
>
>
>
> On Tue, Jul 30, 2013 at 4:58 AM, Antoine Musso  wrote:
>
> > Le 30/07/13 06:39, Ori Livneh a écrit :
> > > There's a short, 1-minute screencast up at
> > > http://ascii.io/a/4428demonstrating usage. Check it out. I'll wait.
> >
> > I am sorry to report you are definitely too awesome.  Thank you for all
> > your work on that front \O/
> >
> > --
> > Antoine "hashar" Musso
> >
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Voting disabled in bugzilla for some products

2013-07-30 Thread Lydia Pintscher 
On Tue, Jul 30, 2013 at 4:17 PM, Andre Klapper  wrote:
> So I assume that you go to
> https://bugzilla.wikimedia.org/query.cgi?query_format=advanced , select
> the components that you maintain and only include tickets that are in
> open state, retrieve the search results (list of open tickets), and sort
> the results by the "Votes" column?

I personally filter for all open bugs that have
wikidata-b...@lists.wikimedia.org as assignee or cc and then sort by
vote, yes. I have a saved search for that. I look at these roughly
weekly to see if this still matches what I hear elsewhere in the
community. I do not only take votes into account because of course a
lot of people don't use bugzilla and even fewer vote there. In
addition I've heard a few too many "I'm not voting because it doesn't
change anything anyway" for my taste. Guess that's a matter of
training of the past years -.-


Cheers
Lydia

-- 
Lydia Pintscher - http://about.me/lydia.pintscher
Community Communications for Technical Projects

Wikimedia Deutschland e.V.
Obentrautstr. 72
10963 Berlin
www.wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.

Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das
Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Is assert() allowed?

2013-07-30 Thread Max Semenik 
I remeber we discussed using asserts and decided they're a bad idea
for WMF-deployed code - yet I see

Warning:  assert() [function.assert]: Assertion 
failed in 
/usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
 on line 291

Thoughts?


-- 
Best regards,
  Max Semenik ([[User:MaxSem]])


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tyler Romeo 
On Tue, Jul 30, 2013 at 5:28 PM, Max Semenik  wrote:

> Warning:  assert() [function.assert]:
> Assertion failed in
> /usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
> on line 291
>

Is this on a production server? If so then the even more confusing question
would be why assertions are enabled on an enterprise system...

As for whether MW should use assertions, I don't remember/wasn't there for
the original discussion, so I can't comment on that, although personally I
don't see how they're that bad an idea.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] "You have new messages" glitch?

2013-07-30 Thread Steve Summit 
Today I'm noticing that if I visit someone else's user or talk
page (this is on en.wp), I see a little orange box saying
"Talk: you have new messages" even though I don't.  Presumably
that user does, or something.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] "You have new messages" glitch?

2013-07-30 Thread Benny Situ 
It's caused by this change: https://gerrit.wikimedia.org/r/#/c/74666/, it
has been reverted now.


On Tue, Jul 30, 2013 at 2:58 PM, Steve Summit  wrote:

> Today I'm noticing that if I visit someone else's user or talk
> page (this is on en.wp), I see a little orange box saying
> "Talk: you have new messages" even though I don't.  Presumably
> that user does, or something.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Matthew Walker 
>
> As for whether MW should use assertions, I don't remember/wasn't there for
> the original discussion, so I can't comment on that, although personally I
> don't see how they're that bad an idea.


I wasn't there either; but from my experience assertions are bad because
you are using them to guard for unexpected behavior and to terminate upon
detection. This is, usually, unexpected. IMO throw an exception if you're
going to do this checking; then you at least get the stack trace (and
function arguments!) for the last chance error handler; not to mention
you're giving upstream code the chance to catch it in case it can be
handled.

~Matt Walker
Wikimedia Foundation
Fundraising Technology Team


On Tue, Jul 30, 2013 at 2:30 PM, Tyler Romeo  wrote:

> On Tue, Jul 30, 2013 at 5:28 PM, Max Semenik 
> wrote:
>
> > Warning:  assert() [function.assert]:
> > Assertion failed in
> >
> /usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
> > on line 291
> >
>
> Is this on a production server? If so then the even more confusing question
> would be why assertions are enabled on an enterprise system...
>
> As for whether MW should use assertions, I don't remember/wasn't there for
> the original discussion, so I can't comment on that, although personally I
> don't see how they're that bad an idea.
>
> *-- *
> *Tyler Romeo*
> Stevens Institute of Technology, Class of 2016
> Major in Computer Science
> www.whizkidztech.com | tylerro...@gmail.com
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tim Starling 
On 31/07/13 07:28, Max Semenik wrote:
> I remeber we discussed using asserts and decided they're a bad
> idea for WMF-deployed code - yet I see
> 
> Warning:  assert() [function.assert]:
> Assertion failed in
> /usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
> on line 291

The original discussion is here:



Judge for yourself.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Varnish

2013-07-30 Thread Jacobo Nájera 
Thanks!

I am studying around Varnish and Wikimedia.

Jacobo


On 26/07/13 00:37, Dmitriy Sintsov wrote:
> 
>> 26 Июль 2013 г. 5:59:57 пользователь Jacobo Nájera
>> (jac...@metahumano.org) написал:
>>
>> I am exploring around Varnish Cache, do you know some tools for test
>> performance and benchmarking?
>> Thanks,
>> Jacobo
>>
> https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html
> 
> 
> varnishtop -i txurl
> varnishlog -c -m 'RxURL:^/foo/bar
> 
> where /foo/bar is script path
> 
> Dmitriy


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tyler Romeo 
I think the real issue here is just that assertions sometimes aren't used
correctly.

Assertions and exceptions are fundamentally different concepts. Assertions
should be used for statements that literally should always be true. And I
mean that almost mathematically, as in most assertions should be able to be
logically proven. This is why they can be turned off on production servers,
because they simply won't happen.

Exceptions are just what the name says: exceptions. While they shouldn't
happen often, exceptions do happen, and thus need to be caught and handled.

Also, assertions in PHP do not have any performance overhead once they're
turned off for production servers, so that won't be an issue either.


*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com


On Tue, Jul 30, 2013 at 6:28 PM, Tim Starling wrote:

> On 31/07/13 07:28, Max Semenik wrote:
> > I remeber we discussed using asserts and decided they're a bad
> > idea for WMF-deployed code - yet I see
> >
> > Warning:  assert() [function.assert]:
> > Assertion failed in
> >
> /usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
> > on line 291
>
> The original discussion is here:
>
> <
> http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/59620
> >
>
> Judge for yourself.
>
> -- Tim Starling
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] MediaWiki-Vagrant: new features

2013-07-30 Thread Ori Livneh 
On Tue, Jul 30, 2013 at 9:58 AM, Tyler Romeo  wrote:

> Would there be any way to have a quick method of switching PHP versions,
> specifically for testing purposes? Like "vagrant enable-role php5.4" or
> maybe it'd have it's own command.
>



> On Tue, Jul 30, 2013 at 12:51 PM, Yuri Astrakhan
> wrote:
> > Can we have varnish through a role please? We will have to do a lot of
> dev
> > & test for it soon.
>


Both of these requests sound reasonable. Can you file bugs (severity:
enhancement) in BZ?
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Kevin Israel 
On 07/30/2013 06:28 PM, Tim Starling wrote:
> On 31/07/13 07:28, Max Semenik wrote:
>> I remeber we discussed using asserts and decided they're a bad
>> idea for WMF-deployed code - yet I see
>>
>> Warning:  assert() [function.assert]:
>> Assertion failed in
>> /usr/local/apache/common-local/php-1.22wmf12/extensions/WikibaseDataModel/DataModel/Claim/Claims.php
>> on line 291
> 
> The original discussion is here:
> 
> 
> 
> Judge for yourself.

I'll further elaborate on the "[...] you have to put the source code
inside a string [...]" part. From the [documentation][1]:

> If the assertion is given as a string it will be evaluated as PHP
> code by assert().

As in: that function is just as evil as eval(), and the innocent looking

assert( "$_GET[id] > 0" );

can actually be a security vulnerability, depending on server
configuration (yes, servers can be and are misconfigured). And when
assert() is used like this (yes, there actually is one of these in
WikibaseDataModel):

assert( $this->functionFromSuperclass() );

it might be necessary to check multiple files to verify that a string
is not passed to assert().

Perhaps it might make sense to do

assert( (bool)( ... ) );

though, as pointed out previously, this really is no better than, say:

if ( !( ... ) ) {
throw new MWException( '...' );
}

[1]: http://php.net/manual/en/function.assert.php

-- 
Kevin Israel - MediaWiki developer, Wikipedia editor
http://en.wikipedia.org/wiki/User:PleaseStand

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tim Starling 
On 31/07/13 08:45, Tyler Romeo wrote:
> Assertions and exceptions are fundamentally different concepts. Assertions
> should be used for statements that literally should always be true. And I
> mean that almost mathematically, as in most assertions should be able to be
> logically proven. This is why they can be turned off on production servers,
> because they simply won't happen.

Interesting concept. I think in C, they are most often used for
validating function input, so obviously they can be hit. The Wikipedia
articles [[Assertion (software development)]] and [[Precondition]]
both mention this usage.

In the Wikidata code in question, assertions are used for both
preconditions and postconditions of non-private functions.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tyler Romeo 
On Tue, Jul 30, 2013 at 7:37 PM, Kevin Israel  wrote:
>
>  As in: that function is just as evil as eval(), and the innocent looking
>
> assert( "$_GET[id] > 0" );
>
> assert( $this->functionFromSuperclass() );
>
>
This is what I mean by misusing the assert function. Assert should always
be called by passing a single-quoted string as an argument. If used
correctly, it is no more a security vulnerability than if you were to put
the same code into an if statement.

Also, like I said, assertions are for statements that are always true, so
checking user input with assertions is incorrect.

Interesting concept. I think in C, they are most often used for
> validating function input, so obviously they can be hit. The Wikipedia
> articles [[Assertion (software development)]] and [[Precondition]]
> both mention this usage.


Using assertions to validate function input is indeed a valid usage, but it
should be done in ways where they won't be hit. In other words, they should
not be used for data validation; they should be used in cases where *the
program expects the data to already be valid*.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Jeroen De Dauw 
Hey,

> Assertions should be used for statements that literally should always be
true.

Indeed. This is the case for the assertion that is failing. Apparently
there is a bug somewhere, and a lack of appropriate tests. But never mind
that bug, we got a much more exciting flame war to keep going here :)

IMO throw an exception if you're
> going to do this checking; then you at least get the stack trace (and
> function arguments!) for the last chance error handler; not to mention
> you're giving upstream code the chance to catch it in case it can be
> handled.
>

Agreed. Exceptions are generally better. Assertions are the lazy way out,
just like type hints. Type hint violations do pretty much the same thing as
assertion violations. Arguing against asserts while not minding type hints
is odd, as the later is arguably more of an issue, as these ones causes
issues on invalid input. Putting in a pile of ifs that do instanceof checks
and whatnot does cause clutter.

So as with pretty much everything in the field of engineering, think about
the tradeoffs of the various approaches whenever you need to pick one.
Deciding on one and then religiously following it everywhere is going to
end poorly, no matter which approach you pick.

ps. Exception handling is generally done quite badly in MW core and
extensions. See
http://sourceforge.net/mailarchive/message.php?msg_id=31231379

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Is assert() allowed?

2013-07-30 Thread Tim Starling 
On 31/07/13 09:46, Tyler Romeo wrote:
>> Interesting concept. I think in C, they are most often used for
>> validating function input, so obviously they can be hit. The Wikipedia
>> articles [[Assertion (software development)]] and [[Precondition]]
>> both mention this usage.
> 
> 
> Using assertions to validate function input is indeed a valid usage, but it
> should be done in ways where they won't be hit. In other words, they should
> not be used for data validation; they should be used in cases where *the
> program expects the data to already be valid*.

I think exceptions should be used for that. Like I said in 2012, the
implementation of assertions in PHP has lots of problems.

You said previously:
> This is why they can be turned off on production servers,
> because they simply won't happen.

You can't mathematically prove that the behaviour of future developers
will follow your expectations. Assertions intended to enforce
developer behaviour are routinely hit in production. It's not correct
to assume that unit testing will discover all bugs, and that
production code will be perfect.

> Also, assertions in PHP do not have any performance overhead once they're
> turned off for production servers, so that won't be an issue either.

That's only the case when the code is encoded as a string, which it's
not in Wikibase. And as PleaseStand points out, the fact that assert()
can act like eval() can make it hard to verify that code is not an
arbitrary script execution vulnerability.

Note that the current thread is about a bug discovered by an assertion
logged in production. If we disabled assertions in production, we
would not know about it.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l