Re: [Wikitech-l] ANN: Version 2.0.0 of the Memento MediaWiki Extension

2014-06-17 Thread Benjamin Lees
Congratulations on this milestone, Shawn, and thanks to you and your team
for all your hard work. :)


On Sat, Jun 14, 2014 at 10:21 PM, Shawn Jones sj...@cs.odu.edu wrote:

 All,

 Thanks to the feedback of developers from this list (wikitech-l), version
 2.0.0 of the Memento time travel Extension for MediaWiki has been released.
  The extension can be downloaded via [1].  Information on the extension is
 available at [2].  A demonstration wiki equipped with the extension is
 available at [3].

 The extension works with Memento clients [4] to allow one to browse pages
 on an installation of MediaWiki as if it were some datetime in the past.
  This has numerous applications, from avoiding spoilers [5] to studying the
 changes in law.

 Recently, at WikiConference USA 2014, we presented our experiences
 reconstructing the past using MediaWiki [6, 7], and demonstrated using the
 extension to avoid spoilers [8, 9].

 The extension is fully compliant with RFC 7089 [10], which specifies the
 Memento protocol.  The effort was supported in part by the Andrew W. Mellon
 Foundation and is a joint effort between Old Dominion University and Los
 Alamos National Laboratory.

 The Memento protocol extends HTTP to support content negotiation with
 time, allowing one to view past versions of web pages.  This document [11]
 provides insight into the added value that the protocol brings to
 MediaWiki.  Videos [12], and [13] show Memento at work in the web at large.
  The latter pays significant attention to temporal navigation in Wikipedia,
 and hence illustrates that the Memento Team considers time travel in
 MediaWiki platforms a major use case of the protocol.  Video [14] shows how
 one could use the Memento MediaWiki Extension to avoid spoilers in
 fan-based wikis.

 The Memento protocol is currently used by major web archives [15] and
 supported by the International Internet Preservation Consortium [16].

 Once again, the Memento team is appreciative of all of the feedback from
 the Wikimedia team, and just because we have reached 2.0.0 doesn’t mean
 that we don’t seek additional input in order to improve the extension.

 On behalf of the Memento Team,

 Shawn M. Jones
 Graduate Research Assistant
 Department of Computer Science
 Old Dominion University

 Email:  sj...@cs.odu.edu
 Research group: http://ws-dl.blogspot.com
 Twitter:@shawnmjones

 
 [1] https://github.com/hariharshankar/mediawiki/releases/tag/v2.0.0
 [2] https://www.mediawiki.org/wiki/Extension:Memento
 [3] http://ws-dl-05.cs.odu.edu/demo/
 [4] http://bit.ly/memento-for-chrome
 [5]
 http://ws-dl.blogspot.com/2013/12/2013-12-18-avoiding-spoilers-with.html
 [6]
 http://wikiconferenceusa.org/wiki/Submissions:Reconstructing_the_past_with_Mediawiki:_Programmatic_Issues_and_Solutions
 [7]
 http://www.slideshare.net/shawnmjones/reconstructing-the-past-with-media-wiki-35333484
 [8]
 http://wikiconferenceusa.org/wiki/Submissions:Using_the_Memento_Mediawiki_Extension_to_Avoid_Spoilers
 [9]
 http://www.slideshare.net/shawnmjones/using-the-memento-mediawiki-extension-to-avoid-spoilers-35333526
 [10] http://tools.ietf.org/html/rfc7089
 [11] http://www.mementoweb.org/wikipedia/
 [12] http://www.youtube.com/watch?v=0_70lQPOOIg
 [13] http://www.youtube.com/watch?v=WtZHKeFwjzk
 [14] https://www.youtube.com/watch?v=ciClYjTnscs
 [15] http://mementoweb.org/depot/
 [16] http://netpreserve.org
 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Getting phpunit working with Vagrant

2014-06-17 Thread Martijn Hoekstra
On Sat, Jun 14, 2014 at 1:10 AM, Dan Duvall dduv...@wikimedia.org wrote:

  would also work, but on my system just `vagrant status` takes 1s which
 is
  not horrible,
  but is noticeable when run  so often. For that reason i went for a check
  that could operate
  without booting ruby and loading all the vagrant code.

 Great point.

 We've been discussing further improvements we might make to
 mediawiki-vagrant, with the primary focus being easier TDD workflows. I'd
 love any other feedback on how to make things more efficient/less painful.
 I'm hoping to fix `vagrant run-tests [extension]` and implement a new
 `vagrant run-browser-tests` in the near future, so if anyone has special
 use cases, let me know.


One thing to keep in mind is non Unixy (read: Windows) hosts. It might not
always be easy for Windows hosts to run PHP-unit with a symlink. I must say
I haven't invested more than an hour or two in getting things to work, and
I'm absolutely *terrible* when it comes to administration tasks, but the
only way I could get the test suite running was with the Pear PHP-unit
version from the guest system, and wasn't able to run the full test-suite
due to OOM conditions (I increased guest memory up to 3 GB).

--Martijn



 On Fri, Jun 13, 2014 at 3:27 PM, Erik Bernhardson 
 ebernhard...@wikimedia.org wrote:

  On Fri, Jun 13, 2014 at 3:04 PM, Dan Duvall dduv...@wikimedia.org
 wrote:
 
   
I use a horrible hack, essentially we run the tests from a pre-commit
   hook
with `make phpunit`, but we redefine make as:
   
  
   This might be naive of me, given I don't know specifically how you
  maintain
   your hooks, but why not modify the pre-commit hook to conditionally
  execute
   over the vagrant ssh tunnel instead of hijacking make?
  
   would probably work as well.  When i wrote this script i already had
 the
  make() stuff
  in my .bashrc so i could also use make from the command line. The
  pre-commit script[1]
  was written based off that, just looks at the commit to figure out which
  make commands to
  run, then calls make.
 
   [1]
 
 
 https://github.com/wikimedia/mediawiki-extensions-Flow/blob/master/scripts/pre-review
 
  Also, instead of crawling parent directories for the Vagrantfile, you
 could
   use `if (which vagrant  vagrant status)  /dev/null; then ...`.
  
  
  would also work, but on my system just `vagrant status` takes 1s which
 is
  not horrible,
  but is noticeable when run  so often. For that reason i went for a check
  that could operate
  without booting ruby and loading all the vagrant code.
 
  Cheers,
   Dan
  
   On Fri, Jun 13, 2014 at 2:35 PM, Erik Bernhardson 
   ebernhard...@wikimedia.org wrote:
  
On Fri, Jun 13, 2014 at 12:49 PM, Jon Robson jdlrob...@gmail.com
   wrote:

 I would like to run the phpunit tests from inside my host OS
  though...
 I have pre-commit hook I rely on to run phpunit tests before
 submitting... how can I do that now?


I use a horrible hack, essentially we run the tests from a pre-commit
   hook
with `make phpunit`, but we redefine make as:
   
make() {
if is_vagrant; then
vagrant ssh -- cd /vagrant/mediawiki/extensions/Flow ''
 sudo
  su
www-data -c 'make $*' || exit 1
else
/usr/bin/env make $* || exit 1
fi
}
   
where the is_vagrant function duplicates the vagrant functionality of
looking at all parent directories for a Vagrantfile
   
   
   
 On Fri, Jun 13, 2014 at 11:45 AM, Dan Duvall 
 dduv...@wikimedia.org
 wrote:
  Jon,
 
  From the looks of it, you may be invoking the tests from your
 host
   OS,
 not
  the Vagrant-managed VM. Trying logging in to the VM using
 `vagrant
   ssh`
 and
  executing the tests from there.
 
master x ~/git/vagrant $ vagrant ssh
...
vagrant@mediawiki-vagrant:~$ cd
 /vagrant/mediawiki/tests/phpunit
vagrant@mediawiki-vagrant:/vagrant/mediawiki/tests/phpunit$
 php
  phpunit.php
 
  If you still have problems with it, feel free to come grab me and
  we
can
  troubleshoot it further.
 
  On a related note, I'll be working on improving the
  mediawiki-vagrant
  browser tests setup for MobileFrontend in the coming weeks. It'd
 be
great
  to have you, or someone else on the mobile team, vet the
   improvements.
 
  Cheers,
  Dan
 
  --
  Dan Duvall
  Automation Engineer
  Wikimedia Foundation http://wikimediafoundation.org
  ___
  Wikitech-l mailing list
  Wikitech-l@lists.wikimedia.org
  https://lists.wikimedia.org/mailman/listinfo/wikitech-l



 --
 Jon Robson
 * http://jonrobson.me.uk
 * https://www.facebook.com/jonrobson
 * @rakugojon

 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 

[Wikitech-l] Bugday on older MediaWiki bugs with high priority set on Tue, June 24 2014, 17:00UTC

2014-06-17 Thread Andre Klapper
Hi everybody,

you are invited to join us on the next Bugday:

 Tuesday, June 24, 2014, 17:00 to 18:30UTC [1]
in #wikimedia-office on Freenode IRC [2]

We will be triaging open Bugzilla tickets under the product MediaWiki
which have high priority set for more than one year.

Everyone is welcome to join, and no technical knowledge needed! It's an
easy way to get involved or to give something back.

All information can be found here:
  https://www.mediawiki.org/wiki/Bug_management/Triage/20140624

For more information on triaging in general and what that means, check
out  https://www.mediawiki.org/wiki/Bug_management/Triage

See you there?

andre


[1] Timezone converter: http://www.timeanddate.com/worldclock/converter.html
[2] See http://meta.wikimedia.org/wiki/IRC for more info on IRC chat
-- 
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] LiquidThreads - how do we kill it?

2014-06-17 Thread S Page
On Mon, Jun 9, 2014 at 11:33 AM, Brad Jorsch (Anomie) bjor...@wikimedia.org
 wrote:


 I personally find the topic history page[4] to be horrendous, both ugly
 and nearly unusable.

We're going to revise topic history.


 Yes, I'm probably atypical in that I like reading wikitext diffs for
 discussion pages.

You can view the diff between revisions of a post, but editing posts is
somewhat rare. Generating a diff of an entire topic with new posts,
updates, hide operations, etc. seems challenging.

-- 
=S Page  Features engineer
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to Keep Track of Template Values Across a Wikipedia?

2014-06-17 Thread Maximilian Klein
Ricordisamoa,

That's a clever idea. It's not exactly what the Abuse Filter is for, but I
made a request for that tag anyway. Thanks for the idea.

Max Klein
‽ http://notconfusing.com/


On Sat, Jun 14, 2014 at 12:02 PM, Ricordisamoa ricordisa...@openmailbox.org
 wrote:

 If the Abuse filter is smart enough to detect every edit that changes a
 value in those templates, it can attach a tag to those edits. Then, it
 would be easy to follow recent changes for that tag.

 Il 13/06/2014 23:53, Maximilian Klein ha scritto:

  Hello All,

 I'm working on the Open-Access Signalling Project[1], which aims to signal
 and badge when a reference in Wikipedia is Open Access source. I'm writing
 the bot at the moment to do this, and I'm encountering a question - how do
 I keep track of the values of the template {{Cite doi | doi=value}}, in as
 close to real-time as possible?

 The most efficient approach I can come up with is to query the SQL servers
 on Labs in constant loop, returning the results of What transcludes
 {{Cite
 doi}} and seeing if the last_edited timestamp is newer than previous? If
 the last_edit is newer, then get the content of the page and see if the
 {{Cite_doi}} value has changed, checking against a local database.

 This seems horribly inefficient still. Is there a hook to know when a
 template on a page has been edited, rather than having to check every time
 the page has been edited?

 Thanks in advance,

 Max Klein
 ‽ http://notconfusing.com/

 [1]
 https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Open_
 Access/Signalling_OA-ness

 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to Keep Track of Template Values Across a Wikipedia?

2014-06-17 Thread Maximilian Klein
John, and John,

Using the External URL search is a good idea. I think I'm going to do this
on the database replicas too.

Thanks for the tips.

Max Klein
‽ http://notconfusing.com/


On Mon, Jun 16, 2014 at 12:25 PM, John phoenixoverr...@gmail.com wrote:

 Ok, taking a closer look all you need to do is to track external link
 usage. It appears that all the cite templates use a
 http://dx.doi.org/XX format URL.

 write a program to parse/keep track of the uses It shouldnt be that hard.
 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Matthew Walker
Today, Kaldari wanted to have translate enabled on foundationwiki so that
Special:MyLanguage was available, but that would fall afoul of bug:44871
[1]. This is not a unique request, fundraising also has some use for
MyLanguage features on wikis that don't have (and wont have) translate.

Are there any concerns with separating out that special page into it's own
extension? The one that immediately jumps to mind is how the i18n team
bundles translate.

[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=44871

~Matt Walker
Wikimedia Foundation
Fundraising Technology Team
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Max Semenik
The page looks kinda small and useful, so why not just move it into core?


On Tue, Jun 17, 2014 at 4:20 PM, Matthew Walker mwal...@wikimedia.org
wrote:

 Today, Kaldari wanted to have translate enabled on foundationwiki so that
 Special:MyLanguage was available, but that would fall afoul of bug:44871
 [1]. This is not a unique request, fundraising also has some use for
 MyLanguage features on wikis that don't have (and wont have) translate.

 Are there any concerns with separating out that special page into it's own
 extension? The one that immediately jumps to mind is how the i18n team
 bundles translate.

 [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=44871

 ~Matt Walker
 Wikimedia Foundation
 Fundraising Technology Team
 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
Best regards,
Max Semenik ([[User:MaxSem]])
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Ryan Kaldari
On Tue, Jun 17, 2014 at 4:20 PM, Matthew Walker mwal...@wikimedia.org
wrote:

 Today, Kaldari wanted to have translate enabled on foundationwiki so that
 Special:MyLanguage was available, but that would fall afoul of bug:44871
 [1]. This is not a unique request, fundraising also has some use for
 MyLanguage features on wikis that don't have (and wont have) translate.

 Are there any concerns with separating out that special page into it's own
 extension?


Or even better, moving Special:MyLanguage into core.

Ryan Kaldari
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread MZMcBride
Matthew Walker wrote:
Today, Kaldari wanted to have translate enabled on foundationwiki so that
Special:MyLanguage was available, but that would fall afoul of bug:44871
[1]. This is not a unique request, fundraising also has some use for
MyLanguage features on wikis that don't have (and wont have) translate.

Are there any concerns with separating out that special page into it's own
extension? The one that immediately jumps to mind is how the i18n team
bundles translate.

[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=44871

Hi.

I read your post and
https://bugzilla.wikimedia.org/show_bug.cgi?id=44871#c19 and still can't
figure out what Kaldari wants to do. Can someone explain why
Special:MyLanguage is needed on wikimediafoundation.org?

I'm also a little confused about the reference to Extension:Collection
in the subject line. Did you mean Extension:Translate? (Also, its. :P)

Putting Special:MyLanguage-type functionality in MediaWiki core seems
fine to me. It just maps a user's language preference to a language code
subpage, right? So Special:MyLanguage/Foo for someone who's specified
that she speaks Spanish would redirect to Foo/es.

MZMcBride



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Ryan Kaldari
The specific need is that I want to set the 'Mobile-frontend-terms-url'
message to '//m.wikimediafoundation.org/wiki/Special:MyLanguage/Terms_of_Use'
via the WikimediaMessages extension, so that people clicking the 'Terms of
Use' link at the bottom of the page will get sent to the Terms of Use in
their own language. Right now, wiki administrators have to override this
message locally to point to the correct translation (after checking to see
if it actually exists).

(Added this information to the bug as well.)

Ryan Kaldari


On Tue, Jun 17, 2014 at 5:49 PM, MZMcBride z...@mzmcbride.com wrote:

 Matthew Walker wrote:
 Today, Kaldari wanted to have translate enabled on foundationwiki so that
 Special:MyLanguage was available, but that would fall afoul of bug:44871
 [1]. This is not a unique request, fundraising also has some use for
 MyLanguage features on wikis that don't have (and wont have) translate.
 
 Are there any concerns with separating out that special page into it's own
 extension? The one that immediately jumps to mind is how the i18n team
 bundles translate.
 
 [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=44871

 Hi.

 I read your post and
 https://bugzilla.wikimedia.org/show_bug.cgi?id=44871#c19 and still can't
 figure out what Kaldari wants to do. Can someone explain why
 Special:MyLanguage is needed on wikimediafoundation.org?

 I'm also a little confused about the reference to Extension:Collection
 in the subject line. Did you mean Extension:Translate? (Also, its. :P)

 Putting Special:MyLanguage-type functionality in MediaWiki core seems
 fine to me. It just maps a user's language preference to a language code
 subpage, right? So Special:MyLanguage/Foo for someone who's specified
 that she speaks Spanish would redirect to Foo/es.

 MZMcBride



 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Matthew Walker
On Tue, Jun 17, 2014 at 5:49 PM, MZMcBride z...@mzmcbride.com wrote:

 Can someone explain why Special:MyLanguage is needed on
 wikimediafoundation.org?


I don't know why Kaldari wants it. But Fundraising can use it for our thank
you page. We would like to give a redirect like
wikimediafoundation.org/wiki/Special:MyLanguage/Thank_You/fi and have it go
to the Finnish page, or English if that doesn't exist. Thereby taking some
amount of intelligence out of that process. We also have links in our
emails referencing reports / statements on foundationwiki that currently
just point to the English versions.


 I'm also a little confused about the reference to Extension:Collection
 in the subject line. Did you mean Extension:Translate? (Also, its. :P)


*sigh* Yes... /me apparently has context switching leakage.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Finding images

2014-06-17 Thread Lars Aronsson

Why is it still, now in 2014, so hard to find images?
We have categories and descriptions, but we also know
they don't describe all that we want to find in an
image. If I need an image with a bicycle and some red
flowers, I can only go to the category:bicycles and
hope that I'm lucky when browsing through the first
700 images there. Most likely, the category will be
subdivided by country or in some other useless way
that will make my search harder.

Where is science? Google was created in 1998, based
on its Pagerank algorithm for web pages filled with
words and links. That was 14 years ago. But what
algorithms are there for finding images?


--
  Lars Aronsson (l...@aronsson.se)
  Aronsson Datateknik - http://aronsson.se



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Separating Special:MyLanguage from Extension:Collection

2014-06-17 Thread Matthew Flaschen

On 06/17/2014 08:57 PM, Ryan Kaldari wrote:

The specific need is that I want to set the 'Mobile-frontend-terms-url'
message to '//m.wikimediafoundation.org/wiki/Special:MyLanguage/Terms_of_Use'
via the WikimediaMessages extension, so that people clicking the 'Terms of
Use' link at the bottom of the page will get sent to the Terms of Use in
their own language. Right now, wiki administrators have to override this
message locally to point to the correct translation (after checking to see
if it actually exists).


Side note, extensions can no longer override core messages.  I don't 
know if extensions can override other extensions, but I doubt it.


So you might have to add on to the mechanism I did (reviews welcome, 
especially from i18n people) at 
https://gerrit.wikimedia.org/r/#/c/137528/ (after the patch is merged, 
it's easy to just add one more key).


Matt Flaschen


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread Tim Starling
On 18/06/14 11:13, Lars Aronsson wrote:
 Why is it still, now in 2014, so hard to find images?
 We have categories and descriptions, but we also know
 they don't describe all that we want to find in an
 image. If I need an image with a bicycle and some red
 flowers, I can only go to the category:bicycles and
 hope that I'm lucky when browsing through the first
 700 images there. Most likely, the category will be
 subdivided by country or in some other useless way
 that will make my search harder.
 
 Where is science? Google was created in 1998, based
 on its Pagerank algorithm for web pages filled with
 words and links. That was 14 years ago. But what
 algorithms are there for finding images?

How do the commercial stock agencies do it? They have a much more
similar problem to Commons than Google does.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread Gerard Meijssen
Hoi,
Now that ONLY indicates that stock agencies have a similar problem to
Commons, it does not help finding images or indicates a path we could take
to improve things.

When images are gaining tags as part of the Wikidatification of multi
mediafiles we at least have a way to add multi lingual support and, that
does improve things on what we have today.
Thanks,
 GerardM


On 18 June 2014 03:46, Tim Starling tstarl...@wikimedia.org wrote:

 On 18/06/14 11:13, Lars Aronsson wrote:
  Why is it still, now in 2014, so hard to find images?
  We have categories and descriptions, but we also know
  they don't describe all that we want to find in an
  image. If I need an image with a bicycle and some red
  flowers, I can only go to the category:bicycles and
  hope that I'm lucky when browsing through the first
  700 images there. Most likely, the category will be
  subdivided by country or in some other useless way
  that will make my search harder.
 
  Where is science? Google was created in 1998, based
  on its Pagerank algorithm for web pages filled with
  words and links. That was 14 years ago. But what
  algorithms are there for finding images?

 How do the commercial stock agencies do it? They have a much more
 similar problem to Commons than Google does.

 -- Tim Starling


 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread Rayson Ho
On Tue, Jun 17, 2014 at 9:13 PM, Lars Aronsson l...@aronsson.se wrote:

 Why is it still, now in 2014, so hard to find images?
 We have categories and descriptions, but we also know
 they don't describe all that we want to find in an
 image. If I need an image with a bicycle and some red
 flowers, I can only go to the category:bicycles and
 hope that I'm lucky when browsing through the first
 700 images there. Most likely, the category will be
 subdivided by country or in some other useless way
 that will make my search harder.


Four years ago I requested the Wikimedia Category Flattening feature:

http://marc.info/?l=wikitech-lm=126525308906767

Fast forward back to 2014 and with an additional 1000 high resolution files
uploaded to wikimedia (over 95% of my photos are released into the public
domain -- it's more free than the iStock editorial license), that feature
is still not done. IMO, a better search function for Wikimedia Commons
would be way more useful than the WYSIWYG editor for Wikipedia!

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html






 Where is science? Google was created in 1998, based
 on its Pagerank algorithm for web pages filled with
 words and links. That was 14 years ago. But what
 algorithms are there for finding images?


 --
   Lars Aronsson (l...@aronsson.se)
   Aronsson Datateknik - http://aronsson.se



 ___
 Wikitech-l mailing list
 Wikitech-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread MZMcBride
Lars Aronsson wrote:
Why is it still, now in 2014, so hard to find images?
We have categories and descriptions, but we also know
they don't describe all that we want to find in an
image. If I need an image with a bicycle and some red
flowers, I can only go to the category:bicycles and
hope that I'm lucky when browsing through the first
700 images there. Most likely, the category will be
subdivided by country or in some other useless way
that will make my search harder.

Where is science? Google was created in 1998, based
on its Pagerank algorithm for web pages filled with
words and links. That was 14 years ago. But what
algorithms are there for finding images?

Hi.

Have you tried Special:Search? :-)

There's a very nice category of red flowers:
https://commons.wikimedia.org/wiki/Category:Red_flowers.

If you search for 'incategory:Red flowers', you can find pictures in
only that category. If you search for 'incategory:Red flowers
incategory:Bicycles', you can see the intersection of these two
categories. (No results currently, alas.) Try a search such as
'incategory:Red flowers incategory:Cosmos atrosanguineus' to see the
search actually work (it should return one result currently, 'File:Cosmos
atrosanguineus Choco Mocha.jpg').

Hope that helps.

MZMcBride



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread Lars Aronsson

On 06/18/2014 06:14 AM, MZMcBride wrote:

If you search for 'incategory:Red flowers', you can find pictures in
only that category. If you search for 'incategory:Red flowers
incategory:Bicycles', you can see the intersection of these two
categories. (No results currently, alas.)


This requires that the interesting images have been
categorized as having red flowers. I could just as well
hope that the description text mentions red flowers,
and do a full text search. Both will fail, because this
detailed level of categorization/description is lacking.

Even though this picture is categorized as fruit vendors,
it isn't categorized as apples, bananas, cherries, peaches,
and pears, or paper crates, or string, or mostly shadow
with a little sunshine on a sidewalk. With 21 million files,
how can we reach that level of detail in documentation?
https://commons.wikimedia.org/wiki/File:Still_Life_with_Fruit-Laden_Bike_-_Medina_%28Old_City%29_-_Rabat_-_Morocco.jpg

Here's a bicycle with red flowers, now categorized,
https://commons.wikimedia.org/wiki/File:Xe_%C4%91%E1%BA%A1p_ch%E1%BB%9F_h%C3%A0ng.jpg



--
  Lars Aronsson (l...@aronsson.se)
  Aronsson Datateknik - http://aronsson.se



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding images

2014-06-17 Thread MZMcBride
Lars Aronsson wrote:
On 06/18/2014 06:14 AM, MZMcBride wrote:
 If you search for 'incategory:Red flowers', you can find pictures in
 only that category. If you search for 'incategory:Red flowers
 incategory:Bicycles', you can see the intersection of these two
 categories. (No results currently, alas.)

This requires that the interesting images have been
categorized as having red flowers. I could just as well
hope that the description text mentions red flowers,
and do a full text search. Both will fail, because this
detailed level of categorization/description is lacking.

This doesn't sound like a technical problem to me... can't you just add
the relevant categories? It's a wiki, after all.

Perhaps you're hoping for automatic image recognition? I don't think
computing, as a science, is there yet. I think I read something about
Google and videos of cats, but even the billionaires can't solve this
problem, yet. Sorry.

Even though this picture is categorized as fruit vendors,
it isn't categorized as apples, bananas, cherries, peaches,
and pears, or paper crates, or string, or mostly shadow
with a little sunshine on a sidewalk. With 21 million files,
how can we reach that level of detail in documentation?
[...]

Click edit. Actually, Commons has HotCat enabled, so you can just click
the (+) link, I imagine. What's the issue?

Here's a bicycle with red flowers, now categorized, [...]

Cool, thanks for that.

MZMcBride



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l