Re: [Wikitech-l] Moving a GitHub Pull Request to Gerrit Changeset manually

[Yuvi Panda]

On Thu, May 23, 2013 at 3:17 AM, Tyler Romeo  wrote:
>> def log(s):
>>print s
> :D

The entire codebase could use some cleanup :) Also I had replaced the
prints with appropriate logging sometime in the past, but changed it
back because lazyness.

> Seriously, though, this is pretty awesome. Also good to see the sh module
> getting some use.

Oh yes, sh is awesome!


--
Yuvi Panda T
http://yuvi.in/blog

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [GSoC 2013] Wikidata Entity Suggester prototype

[Nilesh Chakraborty]

I'm taking the demo offline for a few hours. It will be back up again in a
day's time.

Cheers,
Nilesh


On Wed, May 22, 2013 at 6:06 PM, Nilesh Chakraborty wrote:

> Thanks for the idea! I'll add those pretty soon, shouldn't take much
> effort.
>
> Cheers,
> Nilesh
>
>
> On Wed, May 22, 2013 at 5:16 AM, Matthew Flaschen  > wrote:
>
>> On 05/21/2013 05:29 PM, Nilesh Chakraborty wrote:
>> > You can give it a bunch of property IDs, or a bunch of property-value
>> > pairs, or a mix of both; select the the type of recommendation and hit
>> "Get
>> > suggestions!" :) Feedback is much appreciated.
>>
>> It would be good to show the original properties in a separate section
>> on the result screen, so you can compare what you provided to what was
>> suggested.
>>
>> All should be linked to Wikidata for convenience.
>>
>> Matt Flaschen
>>
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
>
>
> --
> A quest eternal, a life so small! So don't just play the guitar, build one.
> You can also email me at cont...@nileshc.com or visit my 
> website
>
>


-- 
A quest eternal, a life so small! So don't just play the guitar, build one.
You can also email me at cont...@nileshc.com or visit my
website
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Querying the database

[Ryan Kaldari]

There's also stat1.wikimedia.org, but I'm not sure who all has access to 
that. I think just WMF staff and WMF-approved researchers, but I could 
be wrong.


Ryan Kaldari

On 5/22/13 2:28 PM, Marc A. Pelletier wrote:

On 05/22/2013 04:03 PM, Tuszynski, Jaroslaw W. wrote:

So are there any non-toolserver based alternatives for database queries?

Well, there are always the Tool Labs[1].  Database replica access is
still experimental/in trial but it works.

-- Marc
[1]
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Database_access

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Fwd: [QA] First post (almost)

[Quim Gil]

Hi, we have now a mailing list dedicated to QA specific topics:

https://lists.wikimedia.org/mailman/listinfo/qa
http://lists.wikimedia.org/pipermail/qa/

Please forward to your colleagues interested in testing and in 
contributing to Wikimedia!


See the rationale and background below.

 Original Message 

Welcome to the QA list!

This mailing list is an umbrella to host people and discussions focusing 
on software quality assurance in all its aspects: exploratory testing, 
browser automation testing, unit testing, continuous integration, the 
beta cluster, bug management and community QA activities.


We hope this list becomes useful to integrate and retain people
primarily interested in testing / QA. We are seeing many people
interested in testing from different angles. Some people are current
developers willing to learn and discuss more about this topic, best
practices, etc. Some are people new to the community that see testing /
QA as a way to contribute in technical tasks other than development.

This specialized list follows the good results offered by precedents
like the Analytics, Design of Editor Engagement lists. wikitech-l
subscribers will still receive the deep & wide QA related announcements. 
No teams comfortable with the status quo will be encouraged to change 
their current communication practices because of this list.


There is a bit more background at
https://bugzilla.wikimedia.org/show_bug.cgi?id=48057

Now, who's next?

--
Quim Gil
Technical Contributor Coordinator @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Moving a GitHub Pull Request to Gerrit Changeset manually

[Tyler Romeo]

> def log(s):
>print s

:D

Seriously, though, this is pretty awesome. Also good to see the sh module
getting some use.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com


On Wed, May 22, 2013 at 4:45 PM, Steven Walling wrote:

> On Wednesday, May 22, 2013, Yuvi Panda wrote:
>
> > Bump! This is now fully automatic! Enabled for the following repos:
> >
> > qa/browsertests
> > extensions/PostEdit
> > extensions/GuidedTour
> > extensions/GettingStarted
> > extensions/EventLogging
> > extensions/MobileFrontend
> >
> > (I've asked the people who are major contributors to the repos and
> > gotten agreement to turn it on)
> >
> > So everytime a pull request is opened on GitHub for their respective
> > repositories, a Gerrit Changeset will appear in about 4-5 seconds.
> > Updating the Pull Request will also update the Gerrit Changeset.
> > Comment Syncing coming soon.
> >
> > Code is at https://github.com/yuvipanda/SuchABot. It runs on Tools
> > Labs. More detailed description at
> > https://www.mediawiki.org/wiki/User:Yuvipanda/G2G
> >
> > With a little bit more experimentation, perhaps we can figure out a
> > nice workflow, while keeping Gerrit our 'Canonical' Repository.
> >
> > Thanks to everyone who helped!
>
>
> I volunteered to be a guinea pig for this, with a pull request to
> GuidedTour. I'm a good proxy for a newbie since I've never actually
> committed new code to Gerrit, even though I review/comment pretty often.
>
> Steven
>
>
>
> >
> > --
> > Yuvi Panda T
> > http://yuvi.in/blog
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org 
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Querying the database

[Marc A. Pelletier]

On 05/22/2013 04:03 PM, Tuszynski, Jaroslaw W. wrote:
> So are there any non-toolserver based alternatives for database queries?

Well, there are always the Tool Labs[1].  Database replica access is
still experimental/in trial but it works.

-- Marc
[1]
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Database_access

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Moving a GitHub Pull Request to Gerrit Changeset manually

[Steven Walling]

On Wednesday, May 22, 2013, Yuvi Panda wrote:

> Bump! This is now fully automatic! Enabled for the following repos:
>
> qa/browsertests
> extensions/PostEdit
> extensions/GuidedTour
> extensions/GettingStarted
> extensions/EventLogging
> extensions/MobileFrontend
>
> (I've asked the people who are major contributors to the repos and
> gotten agreement to turn it on)
>
> So everytime a pull request is opened on GitHub for their respective
> repositories, a Gerrit Changeset will appear in about 4-5 seconds.
> Updating the Pull Request will also update the Gerrit Changeset.
> Comment Syncing coming soon.
>
> Code is at https://github.com/yuvipanda/SuchABot. It runs on Tools
> Labs. More detailed description at
> https://www.mediawiki.org/wiki/User:Yuvipanda/G2G
>
> With a little bit more experimentation, perhaps we can figure out a
> nice workflow, while keeping Gerrit our 'Canonical' Repository.
>
> Thanks to everyone who helped!


I volunteered to be a guinea pig for this, with a pull request to
GuidedTour. I'm a good proxy for a newbie since I've never actually
committed new code to Gerrit, even though I review/comment pretty often.

Steven



>
> --
> Yuvi Panda T
> http://yuvi.in/blog
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org 
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Moving a GitHub Pull Request to Gerrit Changeset manually

[Yuvi Panda]

Bump! This is now fully automatic! Enabled for the following repos:

qa/browsertests
extensions/PostEdit
extensions/GuidedTour
extensions/GettingStarted
extensions/EventLogging
extensions/MobileFrontend

(I've asked the people who are major contributors to the repos and
gotten agreement to turn it on)

So everytime a pull request is opened on GitHub for their respective
repositories, a Gerrit Changeset will appear in about 4-5 seconds.
Updating the Pull Request will also update the Gerrit Changeset.
Comment Syncing coming soon.

Code is at https://github.com/yuvipanda/SuchABot. It runs on Tools
Labs. More detailed description at
https://www.mediawiki.org/wiki/User:Yuvipanda/G2G

With a little bit more experimentation, perhaps we can figure out a
nice workflow, while keeping Gerrit our 'Canonical' Repository.

Thanks to everyone who helped!

--
Yuvi Panda T
http://yuvi.in/blog

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Querying the database

[John]

Feel free to drop me a mail off list, Ive got a TS account and will gladly
lend a hand with the reports


On Wed, May 22, 2013 at 4:24 PM, Petr Onderka  wrote:

> This is probably not what you want to hear, but one way would be to get a
> Toolserver account.
> That way, you wouldn't need the query service, you could run those queries
> by yourself.
>
> Petr Onderka
> [[en:User:Svick]]
>
>
> On Wed, May 22, 2013 at 10:03 PM, Tuszynski, Jaroslaw W. <
> jaroslaw.w.tuszyn...@saic.com> wrote:
>
> > I do a lot of maintenance tasks on Commons, and many tasks require some
> > sort of database query to find the oddball cases. The queries can be done
> > through one of several ways:
> > 1) Using CatScan and CatScan2[2] tools
> > 2) Database query service [3]
> > 3) Weekly Database reports [4]
> >
> > Unfortunately lately some of those ways are breaking down. CatScan and
> > CatScan2 rarely work failing in many different ways: usually due to
> > exceeding the 'max_user_connections' (30 for magnus's CatScan2, and 15
> for
> > Daniel's CatScan), but otherwise with some timeout or no-connection
> errors,
> > or can work on a query for hours (or days if you let it) and never
> returns
> > anything. I developed some CatScan2 based queries for Creator template
> > maintenance, that worked fine 2-3 years ago, but always time-out since.
> > That might be due more and more images on Commons.  Similarly, Database
> > query service seems also very inactive. There are many requests and few
> > replies, like my request from April 2 [5].
> >
> > For example, lately I was searching for images on Commons that do not
> have
> > any license templates (sometimes since 2007 or earlier), see [5]. At some
> > point
> > Magnus was helping me with that query, however after it  failed several
> > times with "server not found" error we gave up. It seems like less and
> less
> > can be done with current infrastructure.
> >
> > So are there any non-toolserver based alternatives for database queries?
> I
> > was trying to read about Wikimedia Labs looking for tools based on them.
> > Ideally there would be some CatScan2 like tool that is based on different
> > database, with higher number of users allowed.
> >
> > Jarek T.
> > User:jarekt [6]
> >
> > [1] http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php
> > [2] http://toolserver.org/~magnus/catscan_rewrite.php
> > [3] https://jira.toolserver.org/browse/DBQ
> > [4] http://commons.wikimedia.org/wiki/Commons:Database_reports
> > [5] https://jira.toolserver.org/browse/DBQ-201
> > [6] http://commons.wikimedia.org/wiki/User:Jarekt
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Querying the database

[Petr Onderka]

This is probably not what you want to hear, but one way would be to get a
Toolserver account.
That way, you wouldn't need the query service, you could run those queries
by yourself.

Petr Onderka
[[en:User:Svick]]


On Wed, May 22, 2013 at 10:03 PM, Tuszynski, Jaroslaw W. <
jaroslaw.w.tuszyn...@saic.com> wrote:

> I do a lot of maintenance tasks on Commons, and many tasks require some
> sort of database query to find the oddball cases. The queries can be done
> through one of several ways:
> 1) Using CatScan and CatScan2[2] tools
> 2) Database query service [3]
> 3) Weekly Database reports [4]
>
> Unfortunately lately some of those ways are breaking down. CatScan and
> CatScan2 rarely work failing in many different ways: usually due to
> exceeding the 'max_user_connections' (30 for magnus's CatScan2, and 15 for
> Daniel's CatScan), but otherwise with some timeout or no-connection errors,
> or can work on a query for hours (or days if you let it) and never returns
> anything. I developed some CatScan2 based queries for Creator template
> maintenance, that worked fine 2-3 years ago, but always time-out since.
> That might be due more and more images on Commons.  Similarly, Database
> query service seems also very inactive. There are many requests and few
> replies, like my request from April 2 [5].
>
> For example, lately I was searching for images on Commons that do not have
> any license templates (sometimes since 2007 or earlier), see [5]. At some
> point
> Magnus was helping me with that query, however after it  failed several
> times with "server not found" error we gave up. It seems like less and less
> can be done with current infrastructure.
>
> So are there any non-toolserver based alternatives for database queries? I
> was trying to read about Wikimedia Labs looking for tools based on them.
> Ideally there would be some CatScan2 like tool that is based on different
> database, with higher number of users allowed.
>
> Jarek T.
> User:jarekt [6]
>
> [1] http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php
> [2] http://toolserver.org/~magnus/catscan_rewrite.php
> [3] https://jira.toolserver.org/browse/DBQ
> [4] http://commons.wikimedia.org/wiki/Commons:Database_reports
> [5] https://jira.toolserver.org/browse/DBQ-201
> [6] http://commons.wikimedia.org/wiki/User:Jarekt
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Tyler Romeo]

On May 22, 2013 3:36 PM, "Daniel Friesen" 
wrote:
>
> On Wed, 22 May 2013 12:31:50 -0700, Jeroen De Dauw 
wrote:
>
>> Hey,
>>
>>> * WebRequest does much more than routing
>>
>>
>> :(
>>
>>> so it's not going to disappear.
>>
>>
>> :((
>>
>> Cheers
>>
>> --
>> Jeroen De Dauw
>> http://www.bn2vs.com
>> Don't panic. Don't be evil.
>> --
>
>
> What's your problem with WebRequest anyways?
>
>
> --
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
>
>
>

Agreed. WebRequest isn't that bad. It could use some cleanup, but overall
I've found it pretty useful.

Also, I didn't realize WebRequest had anything to do with routing at all.
It was my understanding that it is more just an object oriented interface
to $_GET and whatnot.
___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Querying the database

[Tuszynski, Jaroslaw W.]

I do a lot of maintenance tasks on Commons, and many tasks require some sort of 
database query to find the oddball cases. The queries can be done through one 
of several ways:
1) Using CatScan and CatScan2[2] tools
2) Database query service [3]
3) Weekly Database reports [4]

Unfortunately lately some of those ways are breaking down. CatScan and CatScan2 
rarely work failing in many different ways: usually due to exceeding the 
'max_user_connections' (30 for magnus's CatScan2, and 15 for Daniel's CatScan), 
but otherwise with some timeout or no-connection errors, or can work on a query 
for hours (or days if you let it) and never returns anything. I developed some 
CatScan2 based queries for Creator template maintenance, that worked fine 2-3 
years ago, but always time-out since. That might be due more and more images on 
Commons.  Similarly, Database query service seems also very inactive. There are 
many requests and few replies, like my request from April 2 [5]. 

For example, lately I was searching for images on Commons that do not have any 
license templates (sometimes since 2007 or earlier), see [5]. At some point 
Magnus was helping me with that query, however after it  failed several times 
with "server not found" error we gave up. It seems like less and less can be 
done with current infrastructure.

So are there any non-toolserver based alternatives for database queries? I was 
trying to read about Wikimedia Labs looking for tools based on them. Ideally 
there would be some CatScan2 like tool that is based on different database, 
with higher number of users allowed. 

Jarek T.
User:jarekt [6]

[1] http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php 
[2] http://toolserver.org/~magnus/catscan_rewrite.php 
[3] https://jira.toolserver.org/browse/DBQ 
[4] http://commons.wikimedia.org/wiki/Commons:Database_reports 
[5] https://jira.toolserver.org/browse/DBQ-201 
[6] http://commons.wikimedia.org/wiki/User:Jarekt 
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Bugzilla Weekly Report

[Andre Klapper]

On Mon, 2013-05-20 at 03:00 +, reporter wrote:
> Status changes this week
> Reports changed/set to UNCONFIRMED:  0   
[...]

Sorry, I didn't manage to get https://gerrit.wikimedia.org/r/#/c/63998/
merged before the script ran on Monday (issue itself is a stupid mistake
introduced recently by https://gerrit.wikimedia.org/r/#/c/62220/ ).

andre
-- 
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Daniel Friesen]

On Wed, 22 May 2013 12:31:50 -0700, Jeroen De Dauw  
 wrote:



Hey,


* WebRequest does much more than routing


:(


so it's not going to disappear.


:((

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--


What's your problem with WebRequest anyways?

--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Jeroen De Dauw]

Hey,

> * WebRequest does much more than routing

:(

> so it's not going to disappear.

:((

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Daniel Friesen]

On Wed, 22 May 2013 11:17:13 -0700, Antoine Musso   
wrote:



Le 22/05/13 17:57, Daniel Friesen a écrit :
On Wed, 22 May 2013 08:50:51 -0700, Chad   
wrote:



Making entry points isn't straightforward (which is why I suggested
maintenance), but it can be done.

We really need to cleanup the bootstrap process.


https://www.mediawiki.org/wiki/Requests_for_comment/Entrypoint_Routing_and_404_handling


Ok I am hijacking the thread.

http://symfony.com/doc/current/components/routing/introduction.html

:-)

That is a robust standalone lib that would give us proper routing and
let us dish out WebRequest.  You can even define the routes in a YAML
file yu.


* The router does not handle non-routed stuff. So it can't properly handle  
support for both routed paths and routing queries to various entrypoints.

* WebRequest does much more than routing so it's not going to disappear.

I've also been contemplating reverse routing. Defining the rules we  
generate many urls with. Trying to define most of that info inside the  
same place we route paths. To make defining new things like routing for  
the ShortUrl extension, etc... or things like the NamespacePaths extension  
simpler.


Besides making custom path patterns simpler that plan was also to tackle  
our issue with things like non-indexed paths vs. indexed ones.
Right now we blindly send all non-query paths to an indexed path (the  
short url) and all queried paths to non-indexed ones.
While we have some non-query URIs that would be best always non-indexed  
(like Search) and some query URIs that actually should be indexed (like  
query pages beyond the first page; after we add rel=next/prev to the  
category page).
Additionally it would be ideal for our non-indexed path to be a pretty url  
too. There is no reason that our non-indexed urls have to be index.php.


--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Daniel Friesen]

On Wed, 22 May 2013 11:18:29 -0700, Tyler Romeo   
wrote:



On Wed, May 22, 2013 at 1:12 PM, Daniel Friesen
wrote:


Looks like we just need to abstract most of WebStart.php and part of
doMaintenance into something like includes/Start.php to support things  
that

are neither web nor maintenance.



Honestly, why don't we just move that stuff into Setup.php. I mean,  
pretty

much every entry point script includes Setup.php anyway. The only things
that should be done in entry points is stuff that's specific to that  
entry

point.


There are entrypoint specific things that have to be done before Setup.php  
but after other common stuff you'd think of putting inside Setup.php.


Settings can be entrypoint specific. Web start needs to do things like  
setup output buffering before including Setup. etc...


--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Tyler Romeo]

On Wed, May 22, 2013 at 1:12 PM, Daniel Friesen
wrote:

> Looks like we just need to abstract most of WebStart.php and part of
> doMaintenance into something like includes/Start.php to support things that
> are neither web nor maintenance.


Honestly, why don't we just move that stuff into Setup.php. I mean, pretty
much every entry point script includes Setup.php anyway. The only things
that should be done in entry points is stuff that's specific to that entry
point.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Antoine Musso]

Le 22/05/13 17:57, Daniel Friesen a écrit :
> On Wed, 22 May 2013 08:50:51 -0700, Chad  wrote:
> 
>> Making entry points isn't straightforward (which is why I suggested
>> maintenance), but it can be done.
>>
>> We really need to cleanup the bootstrap process.
> 
> https://www.mediawiki.org/wiki/Requests_for_comment/Entrypoint_Routing_and_404_handling

Ok I am hijacking the thread.

http://symfony.com/doc/current/components/routing/introduction.html

:-)

That is a robust standalone lib that would give us proper routing and
let us dish out WebRequest.  You can even define the routes in a YAML
file yu.


-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Daniel Friesen]

On Wed, 22 May 2013 09:47:14 -0700, Jeroen De Dauw  
 wrote:



Hey,

I hacked up something extremely evil which is now working for my usecase:
http://pastebin.com/WcT1MPNv


Looks like we just need to abstract most of WebStart.php and part of  
doMaintenance into something like includes/Start.php to support things  
that are neither web nor maintenance.


--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Jeroen De Dauw]

Hey,

Another problem with this current approach is that it will cause extensions
to be loaded, which I have no use for. This however seems hard to get
around due to LocalSettings both defining needed config and including the
extensions.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Nischay Nahata]

http://www.mediawiki.org/wiki/File:Mediawiki-vagrant-screenshot.png

Does this help?


On Wed, May 22, 2013 at 10:17 PM, Jeroen De Dauw wrote:

> Hey,
>
> I hacked up something extremely evil which is now working for my usecase:
> http://pastebin.com/WcT1MPNv
>
> Clearly this is far from ideal since my code should not know about MW
> internal at all, it should just include some entry point file.
>
> One fun thing I had to do was stuff all local variables in the global
> vraibles list, since some setup files are assuming they are run in global
> scope, and some are assuming they are in the same scope as the other files.
> Since my script is not getting executed in global scope, it is sort of hard
> to load these files in global scope :)
>
> So if someone could create a working solution that has an evilness level of
> less then 9000, I'd be very happy.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil.
> --
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
Cheers,

Nischay Nahata
nischayn22.in
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Jeroen De Dauw]

Hey,

I hacked up something extremely evil which is now working for my usecase:
http://pastebin.com/WcT1MPNv

Clearly this is far from ideal since my code should not know about MW
internal at all, it should just include some entry point file.

One fun thing I had to do was stuff all local variables in the global
vraibles list, since some setup files are assuming they are run in global
scope, and some are assuming they are in the same scope as the other files.
Since my script is not getting executed in global scope, it is sort of hard
to load these files in global scope :)

So if someone could create a working solution that has an evilness level of
less then 9000, I'd be very happy.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Chad]

On Wed, May 22, 2013 at 9:05 AM, Jeroen De Dauw  wrote:
> Hey,
>
> Making entry points isn't straightforward (which is why I suggested
>> maintenance), but it can be done.
>>
>> We really need to cleanup the bootstrap process.
>>
>
> So you are saying it currently is not possible and that a new entry point
> needs to be created?
>
> If so, is there any way to get such a thing done in a short period of time?
> I can throw an hour at it or so, but not more.
>

It seems like it...I'm sorry if I mislead you the other day in trying
to use maintenance, I suppose I just misunderstood your needs.

The pattern in WebStart or Maintenance should give you enough
to go on. I can't promise it's a 1 hour or less task, but it shouldn't
take you too long.

Yeah it sucks, but that just reinforces my point about us needing
to fix this.

FWIW, the request routing stuff is neat, but it's probably overkill
for what we're needing here. Incremental improvements can be
made to the bootstraping if anyone is brave enough to play with
Setup.php ;-)

-Chad

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Jeroen De Dauw]

Hey,

Making entry points isn't straightforward (which is why I suggested
> maintenance), but it can be done.
>
> We really need to cleanup the bootstrap process.
>

So you are saying it currently is not possible and that a new entry point
needs to be created?

If so, is there any way to get such a thing done in a short period of time?
I can throw an hour at it or so, but not more.

What exactly do you mean by "load MediaWiki"?
>

Making it so that I can run code that uses some MediaWiki classes.

What's the use case?
>

Testing of code that depends on MediaWiki.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Daniel Friesen]

On Wed, 22 May 2013 08:50:51 -0700, Chad  wrote:


Making entry points isn't straightforward (which is why I suggested
maintenance), but it can be done.

We really need to cleanup the bootstrap process.


https://www.mediawiki.org/wiki/Requests_for_comment/Entrypoint_Routing_and_404_handling


-Chad


--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Chad]

Making entry points isn't straightforward (which is why I suggested
maintenance), but it can be done.

We really need to cleanup the bootstrap process.

-Chad
On May 22, 2013 8:48 AM, "Tyler Romeo"  wrote:

> What exactly do you mean by "load MediaWiki"? What's the use case? Maybe
> try making your own entry point?
>
> *-- *
> *Tyler Romeo*
> Stevens Institute of Technology, Class of 2015
> Major in Computer Science
> www.whizkidztech.com | tylerro...@gmail.com
>
>
> On Wed, May 22, 2013 at 11:16 AM, Jeroen De Dauw  >wrote:
>
> > Hey,
> >
> > I'd like to load MediaWiki from some PHP script and have not been able to
> > find a way to do this so far.
> >
> > After someone suggested trying to do this as a maintenance script, I
> tried
> > http://pastebin.com/scKFxF2J
> >
> > That however does not work as there is a check in doMaintenance.php
> > (shouldExecute) that makes sure initialization is only done when no
> > function calls are present in the call stack. My script is getting called
> > by some other library and thus there will clearly be function calls in
> the
> > call stack when doMaintenance.php is included. Is it even possible to
> load
> > MediaWiki in such a case? And what is this restriction in shouldExecute
> > good for?
> >
> > Cheers
> >
> > --
> > Jeroen De Dauw
> > http://www.bn2vs.com
> > Don't panic. Don't be evil.
> > --
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Loading MediaWiki from some script

[Tyler Romeo]

What exactly do you mean by "load MediaWiki"? What's the use case? Maybe
try making your own entry point?

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | tylerro...@gmail.com


On Wed, May 22, 2013 at 11:16 AM, Jeroen De Dauw wrote:

> Hey,
>
> I'd like to load MediaWiki from some PHP script and have not been able to
> find a way to do this so far.
>
> After someone suggested trying to do this as a maintenance script, I tried
> http://pastebin.com/scKFxF2J
>
> That however does not work as there is a check in doMaintenance.php
> (shouldExecute) that makes sure initialization is only done when no
> function calls are present in the call stack. My script is getting called
> by some other library and thus there will clearly be function calls in the
> call stack when doMaintenance.php is included. Is it even possible to load
> MediaWiki in such a case? And what is this restriction in shouldExecute
> good for?
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil.
> --
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Loading MediaWiki from some script

[Jeroen De Dauw]

Hey,

I'd like to load MediaWiki from some PHP script and have not been able to
find a way to do this so far.

After someone suggested trying to do this as a maintenance script, I tried
http://pastebin.com/scKFxF2J

That however does not work as there is a check in doMaintenance.php
(shouldExecute) that makes sure initialization is only done when no
function calls are present in the call stack. My script is getting called
by some other library and thus there will clearly be function calls in the
call stack when doMaintenance.php is included. Is it even possible to load
MediaWiki in such a case? And what is this restriction in shouldExecute
good for?

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [GSoC 2013] Wikidata Entity Suggester prototype

[Nilesh Chakraborty]

Thanks for the idea! I'll add those pretty soon, shouldn't take much effort.

Cheers,
Nilesh


On Wed, May 22, 2013 at 5:16 AM, Matthew Flaschen
wrote:

> On 05/21/2013 05:29 PM, Nilesh Chakraborty wrote:
> > You can give it a bunch of property IDs, or a bunch of property-value
> > pairs, or a mix of both; select the the type of recommendation and hit
> "Get
> > suggestions!" :) Feedback is much appreciated.
>
> It would be good to show the original properties in a separate section
> on the result screen, so you can compare what you provided to what was
> suggested.
>
> All should be linked to Wikidata for convenience.
>
> Matt Flaschen
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
A quest eternal, a life so small! So don't just play the guitar, build one.
You can also email me at cont...@nileshc.com or visit my
website
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [GSoC 2013] Wikidata Entity Suggester prototype

[Nilesh Chakraborty]

Hi Denny,

Thanks! I am not sure how accurate it will be, if it doesn't meet
expectations, I might need to think about optimizing the model, different
metrics etc.; I haven't really thought about those at the moment.

> what do the two properties without a value mean here?

Let me explain those. You can take a look at the wiki pages mentioned on
this page - https://github.com/nilesh-c/wikidata-entity-suggester

Currently, things like these are stored on the recommendation engine:
100,32,7
60,151,7
...
56,15210256,12
...

In the first kind you can see, pairs of  and  are there,
along with the relative affinity 7. Now, suppose we have lots of "city"
items and their respective properties. Say someone tries adding another
item that is a city. Now, as he begins adding properties to that item
(properties that generally belong to a city of course), irrespective of
whether he enters any values for them or not, the entity suggester will
suggest "similar" properties. We are not even talking about "values" here.
If the user *does* add values, better recommendations are fetched. This is
primarily about fetching recommendations for "properties"

So, if someone starts adding a new city called "Wonderland" and adds
properties like "is in the administrative
unit"
or "head of local government ",
the suggester will tell the user that probably
"country"
and "flag image " are some
properties that he/she should add. At least that's the idea.

Now, suggesting values - the current implementation of suggesting values is
just a side-addition. It might not be really accurate. What I intend to add
afterwards is something like this: after the user enters stuff like
41,32,45462347, blah blah, he wants "value" suggestions for
property 31, ie. suggesting "values" to properties.

So, in brief, what currently happens:
Suggest "property-value" mappings to new "item".
Suggest "properties" to new "item".
(new item means, anonymous item, an item without an ID, yet to be added)

What I need to add:
Suggest "value" to a "property".
(This is exactly what you were expecting)

In essence, we combine these 3 types of recommendations and do some magic.
I hope this helps you to understand it better. :)

> How quickly are updates processed by the backend, any idea?

On an Intel core i5-2500K quad-core machine with 4G RAM, this dataset (17th
April - 
wikidatawiki-20130417-pages-meta-current.xml.bz2
):

Data points (pairs) - 8360275
Items - 1965516
Properties and Property-Value pairs - 686318

-- took about 45 mins to build the CSV files, and 15-20 mins to build the
Myrrix model. So it's about 1 hour in total. Parallelizing the CSV file
building will probably bring that time down a bit, not certain though.

Adding new data (items, properties etc) at runtime is pretty much
instantaneous - adding a bunch of 1000 data points will probably take 1
sec, adding 10 data points will be 100ms approximately (including the PHP
client's time and all). It's just an estimate from what I've experienced. I
haven't done any proper benchmarks myself.

Cheers,
Nilesh




On Wed, May 22, 2013 at 5:23 PM, Denny Vrandečić <
denny.vrande...@wikimedia.de> wrote:

> Awesome, that looks already pretty promising!
>
> I am not completely sure I understand a few things:
>
> 1074167410
> 106
> 107215627
> 156
>
> what do the two properties without a value mean here?
>
> I would have expected:
>
> 1074167410
> 107215627
>
> and now ask for suggested values for 31,
> or for suggested properties to add.
>
> But these are already details. The results seem pretty promising.
>
> How quickly are updates processed by the backend, any idea?
>
>
>
>
>
>
>
> 2013/5/21 Nilesh Chakraborty 
>
> > Hello,
> >
> > I have some updates on the Entity Suggester prototype. Here are the two
> > repos:
> > 1. https://github.com/nilesh-c/wikidata-entity-suggester
> > 2. https://github.com/nilesh-c/wes-php-client
> >
> > As it stands now, deployment-wise, I have a single Java war file that's
> > deployed on Tomcat. And there's a PHP client that can be used from PHP
> code
> > to push data into or fetch suggestions from that engine.
> >
> > I have made a simple, crude demo that you can access here
> > -http://home.nileshc.com/wesTest.php.
> > You can find the code for it in the wes-php-client repo. It's hosted on
> my
> > home desktop temporarily. I am having some non-technical problems with
> the
> > VPS I'm managing and customer support is working on it. After it starts
> to
> > work, I may try deploying this to the VPS. So, if you have to face an
> > embarrassing 404 page, I'm really sorry, I'll be working on it. If it
> stays
> > up, well and good. :)
> > 
> >
> > You can give

Re: [Wikitech-l] [GSoC 2013] Wikidata Entity Suggester prototype

[Denny Vrandečić]

Awesome, that looks already pretty promising!

I am not completely sure I understand a few things:

1074167410
106
107215627
156

what do the two properties without a value mean here?

I would have expected:

1074167410
107215627

and now ask for suggested values for 31,
or for suggested properties to add.

But these are already details. The results seem pretty promising.

How quickly are updates processed by the backend, any idea?







2013/5/21 Nilesh Chakraborty 

> Hello,
>
> I have some updates on the Entity Suggester prototype. Here are the two
> repos:
> 1. https://github.com/nilesh-c/wikidata-entity-suggester
> 2. https://github.com/nilesh-c/wes-php-client
>
> As it stands now, deployment-wise, I have a single Java war file that's
> deployed on Tomcat. And there's a PHP client that can be used from PHP code
> to push data into or fetch suggestions from that engine.
>
> I have made a simple, crude demo that you can access here
> -http://home.nileshc.com/wesTest.php.
> You can find the code for it in the wes-php-client repo. It's hosted on my
> home desktop temporarily. I am having some non-technical problems with the
> VPS I'm managing and customer support is working on it. After it starts to
> work, I may try deploying this to the VPS. So, if you have to face an
> embarrassing 404 page, I'm really sorry, I'll be working on it. If it stays
> up, well and good. :)
> 
>
> You can give it a bunch of property IDs, or a bunch of property-value
> pairs, or a mix of both; select the the type of recommendation and hit "Get
> suggestions!" :) Feedback is much appreciated.
>
> Cheers,
> Nilesh
>
>
> On Tue, May 14, 2013 at 2:36 AM, Matthew Flaschen
> wrote:
>
> > On 05/13/2013 04:28 PM, Nilesh Chakraborty wrote:
> > > Hi Matt,
> > >
> > > Yes, you're right, they are available as separately licensed downloads.
> > > Only the stand-alone "Serving Layer" is needed for the Entity
> Suggester.
> > > It's licensed under Apache v. 2.0. Since I'm using the software as-is,
> > > without any code modifications, I suppose it's compatible with what
> > > Wikidata would allow?
> >
> > Apache 2.0-licensed software should be fine, even if you do need/want to
> > modify it.
> >
> > Matt Flaschen
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> A quest eternal, a life so small! So don't just play the guitar, build one.
> You can also email me at cont...@nileshc.com or visit my
> website
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



-- 
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l