[SMW-devel] [Security vulnerability] Log4j zero-day exploit info for CirrusSearch and Semantic MediaWiki ElasticStore

[Markus Krötzsch]

Heads up everyone using (S)MW with ElasticSearch!

Markus


 Forwarded Message 
Subject: 	[MediaWiki-l] [Security vulnerability] Log4j zero-day exploit 
info for CirrusSearch and Semantic MediaWiki ElasticStore

Date:   Mon, 13 Dec 2021 13:46:40 -0600
From:   Jeffrey Wang 
Reply-To: 	MediaWiki announcements and site admin list 


To: mediawik...@lists.wikimedia.org

Hello all,

As you may have seen recently, Log4j has a severe zero-day exploit 
affecting many projects, including Elasticsearch. For anyone using 
CirrusSearch or Semantic MediaWiki’s ElasticStore, here’s what you need 
to know:


- If you are using JDK 11 or above, you’re not affected. 😊
- If you are using the latest version of the Elasticsearch 6.x Docker 
images, you’re not affected. This is because 6.6 uses JDK 11, 6.7 uses 
JDK 12, and 6.8 uses JDK 15. 😊
- If you are using JDK 8 or under, you are likely affected. 😭 There are 
a few ways to fix this:
-- First, Elasticsearch 6.8.21 is being released to remedy this. 
Upgrading to this version should resolve the issues even if you are 
using JDK 8 or below.
-- If you are using Elasticsearch 6.5.4, 6.6.x, 6.7.x, or you are 
otherwise unable to upgrade to the latest version of Elasticsearch 6.x, 
I strongly recommend you try upgrading your JDK version to at least JDK 
11 or upgrade Elasticsearch to 6.8.21 when it comes out.
-- If you can’t upgrade your JDK or Elasticsearch, you can set the JVM 
option |Dlog4j2.formatMsgNoLookups=true|


You may have seen information on the CirrusSearch extension page saying 
CirrusSearch 6.5.4 only currently works with Elasticsearch 6.5.4. That 
is not correct; CirrusSearch 6.5.4 works just fine with 6.8.20 (for 
instance, Project Canasta uses the ES 6.8.20 Docker image) and the 
extension page has been updated to reflect that.


For more information from Elastic themselves, please see this:
https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476 



Thanks,
Jeffrey
___
MediaWiki-l mailing list -- mediawik...@lists.wikimedia.org
To unsubscribe send an email to mediawiki-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Maintain order of property values

[Markus Krötzsch]

Hi Michael,

The data model of SMW never considered "insertion order" as a piece of 
information that should be recorded. In fact, a normal SQL RDBMs like 
MariaDB has no obligation to return tuples in a specific order unless 
there is an ordering clause in the query. Same for SPARQL. Relying on 
result orders to be the same "as in the table" is generally not robust.


To store and retrieve ordered lists in SMW, there are several options:

(1) Use sub-objects instead of plain values and store the index of each 
entry with the value. Then sort by this value when retrieving data.
(2) Store the unordered values as you do now, and in addition store a 
string of the ordered values (comma-separated) in another property. You 
can use page variables to build such a string and #arraymap to decompose 
it (if you want to do more than just display this string).


I have worked with both solutions in practice. For things like lecture 
dates, we use (1) (though we use the date rather than the insertion 
order as an ordering criterion there). For things like author lists 
(where order matters), we use (2).


Best regards,

Markus


On 24/09/2020 12:37, Michael Erdmann wrote:

Dear Community,

is there a way to maintain the order of attribute values, e.g. in ASK 
queries. By "maintain" I mean to keep the order of the values as they 
had been added in the wiki page's annotations.


Currently I get this:

Test_Page contains:

[[MyProperty::Page1]]
[[MyProperty::Page2]]
[[MyProperty::Page3]]

Another_Page contains this query:

{{#ask: [[Test_Page]] | ?MyProperty | format = ul }}

and shows this result

Test_Page (MyProperty: Page1, Page2, Page3)

Changing the order in Test_Page to:

[[MyProperty::Page2]]
[[MyProperty::Page3]]
[[MyProperty::Page1]]

still results in exactly the same query result, i.e. Page1 at the 
beginnig of the list.


Is there a way to accomplish what I want in Wiki text?

If there is a way to accomplish this in PHP that is also fine for me. I 
investigated a bit already and found, that


$data = StoreFactory::getStore()->getSemanticData( $myPage );
$values = $data->getPropertyValues( $myProperty );

already returns the values in a constant order (page-IDs?). Is the DB 
maybe not prepared to maintain this information?


BTW. I am using MW1.31 and SMW3.0.2.

THX,
   michael


--

Dr. Michael Erdmann |erdm...@diqa-pm.com| +49 151 6140 1790
DIQA Projektmanagement GmbH | An der RaumFabrik 33c |   76227 Karlsruhe
Handelsregister: Amtsgericht Mannheim HRB 715454 USt-IdNr.: DE283037270
Geschäftsführer: Dr. Michael Erdmann, Dipl.-Wirtsch.-Inf. Daniel Hansch

This email may contain confidential information. If you are not the 
intended recipient please notify the sender immediately and delete this 
email. Any unauthorized copying, disclosure or distribution of this 
email is strictly forbidden.



___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel





___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Question regarding the policy of freezing SMW 3.0 milestone issues

[Markus Krötzsch]

Hi,

In a rare response to this email list, I have to completely back James 
here. The request sounds a bit too much as if we are talking about a 
rescheduling of Ubuntu or some other large project. It is important to 
keep perspective here. For small projects like this, the simple rule is 
that the person(s) who do the work get to make the rules. I certainly 
would not dare to set up a release process for someone else's weekend's 
work!


Now this does not mean that we have to be happy with everything. I think 
David's email was sufficiently carefully phrased and should not be taken 
as an offense -- the exchange helps to clarify where we are all 
standing. The next step is to ask (and this is mainly for those who want 
something) how they can support the project.


I have asked this questions myself a lot when I was still the person 
that got such emails, and I have to admit that I do not have a good 
answer. The transition from a super-tiny/one-person dev team to a small 
size team (with a plan, and processes, and everything) seems very hard. 
I think the natural way is that new devs arrive to help with something, 
get increasingly familiar with the code and core, and at some point are 
part of a growing team. It seems this is not happening with SMW, and it 
would be interesting to know why. Most obviously, PHP/MediaWiki coding 
is not super appealing. But it also may have to do with the extension 
model of SMW and MW: motivated fresh developers tend to make their own 
extension rather than joining another project.


Finally, for those who think that 3.0.0 is done and that open issues 
could be fixed later, I am sure that there is a not-too-hard technical 
solution for using the current development version right away. This 
could also help to improve the release for others.


Cheers,

Markus


On 03/09/18 13:51, James HK wrote:

Hi

Some quick notes:


I share your concerns and the desire to have a 3.0.0 release sooner rather
than later without delaying for more features. We had a quick back and
forth about this at the beginning of June, see
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1365#issuecomment-394930906
and the following comments.


See my response [0].


As you can imagine, there are many of us who are silently and
patiently waiting for MediaWiki 3.0.0 [0] to complete.


I have been silently and patiently waiting for community members to
help fix open tickets some which clearly are non-programming tasks to
move a release forward, yet this hasn't happened especially after [1]
iterates on that very topic.


(I would have loved to help out myself but I currently don't have the free 
time).


I repeatedly have heard this line before and is one of the reasons why
a release is rescheduled or not happen at all.


Version 3.0 was expected to complete around Q4 2017 - Q1 2018. By end
of Q1 it was obvious that was not happening and so it got pushed to Q2
[1] and then to Q3.
More recently, a due date of August 22 was set, then September 2, and
now it's set to September 17 [0].


One thing to remind people of is that this is a volunteer project and
I clarify my position (once more), I'm only doing weekend sprints and
any active development related to it happens then and only then
including code review.

Furthermore, recently MediaWiki core issues showed up (#3397 as a
latest example, a no #3395 it is) in numbers that I'm losing focus and
patience by spending hours during a weekend to fix things that neither
originate nor belong in SMW core.


Is there a general policy/protocol for freezing issues for versions
such as 3.0.0, with the exception of only priority bugs?


No.

If the community thinks it needs such policy then you are free to
discuss that but don't expect any support from my side.


In other words, why is 3.0.0 not frozen for everything but critical
bugs? Why not move many of those issues to 3.1.0?


Because certain things can only be fixed or introduced with a major
release (which is 3.x and not 3.1).


So my question is simple (or complicated): why are new issues still
being added to a milestone that's already a year behind schedule? Let
alone enhancements.


Because the development process is fluid and some features need
readjustment after they have been introduced.

There is no "a year behind schedule" because dates are tentatively set
to give an orientation and not something to be planned around
particular when active participants appear in such a low number.

So, people want to see things happen in a more timely matter then
involvement is paramount because without it changes happen at its own
pace.

[0] 
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1365#issuecomment-402997832.
[1] https://sourceforge.net/p/semediawiki/mailman/message/36248747/

Cheers

On 9/3/18, Jeroen De Dauw  wrote:

Hey David,

I share your concerns and the desire to have a 3.0.0 release sooner rather
than later without delaying for more features. We had a quick back and
forth about

Re: [SMW-devel] SMW 3.0: Rework List format?

[Markus Krötzsch]

Hi,

Just briefly: I support the suggested changes. Much of the current code 
has been kept for legacy reasons and one can surely streamline and 
systematize things now using all the experience gathered over the years.


Cheers,

Markus


On 14.06.2017 22:12, Stephan Gambke via Semediawiki-devel wrote:

Hello.

*TL;DR:* I'd like to rework the List format. Join the discussion at 
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2488.


SMW 3.0 is coming up ([0]). This will be a major new version ([1]) 
allowing to introduce breaking changes. Making such changes should 
certainly not be an end to itself. Still, a major release provides an 
opportunity to introduce new features and clean-up cruft that will 
probably not come too soon again (SMW 2.0 was released three years ago).


With that intro, do you think it would be a good idea to rework the 
|list| format?


Here is what I would like to see:

*Separate separators for values, properties and result "rows".*
E.g. three separators (|sep=,|, |propsep=;|, |valuesep=/|) could produce 
the following result:


United States of America (Cities Washington / New York / Los
Angeles; Neighbors Canada / Mexico), United Kingdom (Cities London /
Glasgow / Manchester; Neighbors Ireland), France (Cities Paris /
Marseille / Bordeaux; Neighbors Belgium / Luxemburg / Germany /
Switzerland / Italy / Spain), and Germany (Cities Berlin / Frankfurt
/ Hamburg; Neighbors Poland / Czechia / Austria / Switzerland /
France / Belgium / Luxemburg / Netherlands / Denmark)

This would also benefit the subformats (|ol|, |ul|, |template|, |tree| 
(in SRF))


*A simpler logic of when which separator is used.*
This is currently handled in ListResultPrinter.php 
 
and is rather confusing. I'd just reduce this to a default value |,|, 
that may be overridden by parser function parameters. The default values 
should be defined in i18n messages.


*Remove the final list separator (", and").*
It has been gone for a while due to a bug and apparently nobody noticed.

*Make the template format an alias of the list format.*
When the user supplies a parser function parameter |template|, then just 
assume they want it used, even for |format=list|. And inversely if 
|format=template|, but the parser function parameter |template| is 
missing or empty, just print a normal list.


*Rename/standardize parameters to templates.*

E.g. all standard parameters start with a |#|:

  * |smw-resultquerycondition| => |#querycondition|
  * |smw-resultquerylimit| => |#querylimit|
  * |smw-resultqueryoffset| => |#resultoffset|
  * new: |#resultcount|

*Add class attribute to HTML elements.*
To facilitate styling.

*General code clean-up and removal of legacy/deprecated functionality*
To make it easier to subclass and test the |list| format.

What else?



Please join the discussion on GitHub 
(https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2488).


Cheers
Stephan


[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/milestone/6
[0] http://semver.org/


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot



___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Semantic MediaWiki 2.5.0 released

[Markus Krötzsch]

Impressive feature list. Congratulations to all SMW devs and 
contributors on this achievement!

Cheers,

Markus


On 15.03.2017 17:08, [[kgh]] wrote:
> Hello everyone,
>
> we are happy to announce the immediate availability of the Semantic
> MediaWiki 2.5.0 release: [0,1]. This is a big release bringing a lot of
> new features and improvements as well as bug fixes.
>
>
> ## Version highlights
>
> * Full-text search in property values introduced
>
> Support for full-text search was added using the native capabilities of
> the SQL backends "MySQL"/"MariaDB" (#1481) and "SQLite" (#1801) for the
> "Text", "URL" and "Page" datatypes.
>
> * Provenance data recording introduced
>
> Qualifying facts using a simple provenance model is now supported
> (#1808) using existing mechanisms in defining a property specification
> together with a new "Reference" datatype.
>
> * Property chain and language filter support in result printouts introduced
>
> Property chains for conditions (e.g [[Located in.Capital of::Foo]]) was
> provided for some time, and now got extended (#1824) to supporting the
> syntax on print requests to retrieve values of a chain member that
> represent a page node. Values of datatype "Monolingual Text" can now use
> a language filter (#2037) to restrict the display of a value in a print
> request.
>
> * Edit protection for better semantic gardening introduced
>
> Edit protection to help avoid changes to properties or other data
> sensitive pages from alterations that may cause data invalidations (e.g.
> change of a property type, inconsistent specifications etc.) or process
> disruptions. This feature integrates with MediaWiki's page protection
> functionality.
>
> * Preferred property label support introduced
>
> Semantic MediaWiki now supports the declaration of preferred property
> labels (#1865) with the objective to show labels in a user context on
> special pages, query results, and factboxes instead of the canonical
> property label.
>
> * Query result cache for better performance introduced
>
> Caching of query results (#1251) was added as experimental feature to
> minimize a possible impact of query processing during and after a page
> view. This change also includes a reevaluation (#2099, #2176) of the
> query hash (used as identifier) to ensure that cache fragmentation is
> reduced and duplicate queries can share the same cache across different
> pages.
>
> * Links in values feature now working reliably
>
> Support for links in values for datatype "Text" was extended by
> use-cases and improved in performance as well as avoiding the former
> error-prone "PCRE-approach".
>
> * Fixed properties now a stable feature
>
> Support for fixed properties was overhauled, fixed (#2135) and is no
> longer experimental.
>
> * Special page "SemanticMediaWiki" formerly "SMWAdmin" overhauled and
> extended
>
> Special page "SemanticMediaWiki" formerly known as special page
> "SMWAdmin" was modernized and extended (#2044, etc.) including a new
> configuration setting allowing for a more fine-granded control over
> feature accessibilty (#2142).
>
>
> See the release notes [0] for these highlights and for much more
> information on the features, enhancements, changes as well as fixes this
> release additionally brings to you.
>
>
> ## Compatibility changes
>
> Semantic MediaWiki 2.5.0 dropped support for PHP 5.4 and lower as well
> as MediaWiki 1.22 and lower. See also the compatibility notes [2] for
> detailed information on supported environments.
>
>
> ## Installing and upgrading
>
> If you are using Semantic MediaWiki via Composer, update the version in
> your "composer.json" or "composer.local.json" file to "~2.5" and run
> "composer update". If you are using the tarballs, you can download them
> here: [1]. Remember to also update your database afterwards by running
> "update.php" due to schema changes and new database tables. If you would
> like to install Semantic MediaWiki you can follow the installation
> instructions: [3].
>
>
> ## Contributors
>
> At this point we would like to thank all people involved making this
> release possible:
>
> James Hong Kong, Karsten Hoffmeyer, Jeroen De Dauw, Niklas Laxström,
> translatewiki.net translators, Maciej Brencz, Felipe de Jong, Siebrand
> Mazeland, Alex Winkler, Stephan Gambke, Toni Hermoso Pulido, Amir E.
> Aharoni, Felipe Schenone, Jaider Andrade Ferreira, James Forrester,
> Justin Du, Sébastien Beyou, Virginia Cepeda.
>
> We also encourage your continued contributions: [4].
>
>
> Thank you to all of you for using Semantic MediaWiki. We wish you a
> pleasant time and a lot of success doing so.
>
> - The SMW development team
>
>
> [0] https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_2.5.0
> [1]
> https://github.com/SemanticMediaWiki/SemanticMediaWiki/releases/tag/2.5.0
> [2]
> https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/docs/COMPATIBILITY.md
> [3]
> https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/docs/I

Re: [SMW-devel] MediaWiki:Smw_uri_blacklist purpose?

[Markus Krötzsch]

Dear Andreas,

The purpose of these was to ensure that the RDF export of SMW is 
well-behaved and also valid in OWL. Many tools would find it hard to 
digest if you would declare a property to be, e.g., a subproperty of 
rdfs:range.

Unfortunately, rdf and rdfs in particular contain a lot of things that 
may not all be problematic. However, rdfs:label, for example, should be 
declared an owl:AnnotationProperty, whereas SMW would declare it as 
owl:DatatypeProperty. Not sure if this is working as expected either.

In general, you can change the blacklist to do whatever you like, but 
since it seems you plan to use a tool that understands rdf and rdfs 
vocabulary (why else care?), you should maybe check if it is really 
working with the RDF output SMW would give you.

Best,

Markus

On 07.09.2016 17:08, Andreas Plank wrote:
> Hi,
>
> what is the use case of MediaWiki:Smw_uri_blacklist
>  http://www.w3.org/1999/02/22-rdf-syntax-ns#
>  http://www.w3.org/2000/01/rdf-schema#
>  http://www.w3.org/2002/07/owl#
> ?
>
> I mean with MediaWiki:Smw_uri_blacklist filled by default, I can not
> define [[some-URI-property::http://www.w3.org/2000/01/rdf-schema#label]]
> or something like it because http://www.w3.org/2000/01/rdf-schema# is in
> the blacklist and then SMW will complain about it and store nothing.
>
> (1) So I want to undersand what would be semantically the proper way ?
> On https://www.semantic-mediawiki.org/wiki/Help:Datentyp_URL (or the
> English version) it just says «But it is not desirable to make new
> statements about those language constructs.» Is there some help
> available (on www.semantic-mediawiki.org
>  ?) on the issue of URIs and the
> blacklist URIs, that I might have overlooked?
>
> (2) So if I want to use rdf:type with MediaWiki:Smw_uri_blacklist filled
> by default
> * I have to set up MediaWiki:Smw_import_rdf
> * I have to set up [[property:rdf:type]] with [[Imported
> from::rdf:type]] on it and specify the [[Has type::Annotation URI]]
> * I have to take care that I set always an URL
> [[rdf:type::http://some-external-url]] or
> [[rdf:type::http://the-local-wiki-url]]
> Is that correct?
>
>
> Thank you for your elaboration or a detailed Help page
> Andreas
>
>
>
>
> --
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Call for Papers Semantic Web

[Markus Krötzsch]

Hi,

I would not consider this journal for publication. Warning signs: 
"Google-based impact factor", special issues on random topics that have 
no clear focus, missing or unknown guest editors, editorial board 
members unknown in respective community, prominent publication fees, 
rather weak papers.

There are several well-established semantic web journals that one can 
choose from, including the Journal of Web Semantics and the Semantic Web 
Journal.

Cheers,

Markus

On 06.07.2016 08:51, Krabina Bernhard wrote:
> Hi,
>
> I don't really know this journal, but they are doing a special issue on 
> Semantic Web. Maybe somebody is interested in publishing somehing SMW related?
>
> Submission Deadline: November 20th, 2016
> Publication Date: January 2017
>
> http://www.scirp.org/journal/htmlOfSpecialIssue.aspx?ID=2998&JournalID=2431&utm_campaign=SpecialIssue&utm_source=e_nl&utm_medium=nl_jcc_sc_20160624_xiangfang_78487#.V3ynDjVdyao
>
> cheers,
> Bernhard
>
> --
> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> present their vision of the future. This family event has something for
> everyone, including kids. Get more information and register today.
> http://sdm.link/attshape
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Set a SMW property programmatically in PHP?

[Markus Krötzsch]

On 29.06.2015 17:16, Yaron Koren wrote:
> Hi Jason,
>
> If you're talking about making a change to the semantic data of a page
> that is not reflected in that page's wikitext, that sounds like a bad
> idea - the new data would be removed as soon as the page was edited and
> saved again.

Indeed, one has to be very careful here. The key is to make the change 
at the correct place to ensure that the data is not lost on updates or 
refreshs, which may not always trigger all MediaWiki hooks that you 
would use during normal parsing. In particular, semantic data is always 
secondary data which comes (mostly) from the content of the page and (in 
rare cases) from other metadata of the page (as for SMW's Modification 
date property).

There are safe ways for setting properties from PHP, especially from 
within own parser functions. SMW's own #set parser function has examples 
on how to do this. Setting special properties like Modification date is 
more tricky.

What you are trying to do sounds dangerous though, since it seems that 
you are trying to set the value of something that is controlled by 
another extension. If this is the case, you should let the other 
extension control it and insert into the page text whatever is necessary 
to get the value that you want. You cannot change SMW data to be 
different from what is actually given in the page -- doing so will only 
cause a temporary inconsistency in the database that will be "fixed" as 
soon as the page is stored/refreshed again. SMW data does not have an 
independent existence or history. It must be regenerated reliably on 
every refresh of the page.

Regards,

Markus

>
> -Yaron
>
> On Thu, Jun 25, 2015 at 3:42 PM, Jason Ji  > wrote:
>
> Hi SMW community,
>
> I have a hopefully simple question. Is there a way to set an SMW
> property for a page programmatically in PHP? Suppose I have access
> to any of the relevant information, like the page ID.
>
> For context, I'm writing an extension which, among other things,
> creates wiki pages using $wikipage->doEditContent(), and I'd like to
> be able to set a semantic title
>  property
> on the page at that point.
>
> Thanks!
>
> --
> Jason Ji
> jason.y...@gmail.com 
>
> 
> --
> Monitor 25 network devices or servers for free with OpManager!
> OpManager is web-based network management software that monitors
> network devices and physical & virtual servers, alerts via email & sms
> for fault. Monitor 25 devices for free with no restriction. Download now
> http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
> --
> Monitor 25 network devices or servers for free with OpManager!
> OpManager is web-based network management software that monitors
> network devices and physical & virtual servers, alerts via email & sms
> for fault. Monitor 25 devices for free with no restriction. Download now
> http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors 
network devices and physical & virtual servers, alerts via email & sms 
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW Ontologies

[Markus Krötzsch]

Hi John,

Sounds interesting, but I don't think I fully understand the 
functionality yet. In particular, how is the data that is thus recorded 
related to the DC vocabulary? For example, dc:title is a property with 
the URI http://purl.org/dc/elements/1.1/title. Is this URI then used in 
the RDF export of SMW? Or do your parser functions include some 
RDFa/microdata to refer to the DC URI?

Regards,

Markus

On 08.05.2015 22:22, John McClure wrote:
> Hi,
> Over the next month or so I am releasing a number of ontologies each as
> an SMW extension. All installed ontologies will be listed on the
> Special:Version page in a new section named "Semantic Ontologies".
>
> For instance one of the ontologies is named "Dublin Core" (whose
> namespace prefix is "dc") - its extension contains just one parser
> function (named "dc") that is used to validate and then set the value(s)
> on a page for any of the properties defined by that ontology. (Please
> see http://dublincore.org/ for official documentation regarding this
> ontology.)
>
> So if you want to set the property "dc:title" for a page, then simply
> call {{dc:title|/here is the formal title of a page/}}. For multilingual
> wikis, one  says {{dc:title|@en=/english formal title/|@de=/german
> title/|...}}; the parser function in this latter case creates a
> subobject named "dc:title" with properties named appropriately eg @en
> and @de.
>
> You may want to create your own property that is a "dc:title" but which
> is to have a different name, here's all you do:
> 1) insert [[ smw:subProperty Of::dc:title]] on (say) your
> "Property:Report name"  page
> 2) call {{dc:title|{{{1|}}}|...|set property=Report name}} on a page or
> within your "class" template
>
> Validation errors are stored in special property "smw:error" that can be
> reported using {{#ask:}}. A property value is not stored if a validation
> error is raised.
>
> A base ontology named "smw" is the first to be released, implemented as
> described above; it contains properties and classes builtin to, or
> implied by, the MediaWiki and Semantic MediaWiki software -- these
> properties and classes form the lowest level ontology upon which ALL
> other classes and properties are then defined (that is, via
> class/subclass and property/subproperty relations).
>
> I am working on an update.php -based mechanism for loading Property and
> Category pages into one's wiki referenced by these ontology parser
> functions. It also loads a page named "Ontology: /name/" into the
> Concept namespace of the wiki so that users may conveniently access
> information about the ontology.
>
> Thanks in advance for your feedback.
> --
> thanks/john
> skype:hypergrove
>
>
> --
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] cargo's page values link

[Markus Krötzsch]

On 06.04.2015 03:57, Yaron Koren wrote:
> I'm guessing that it's possible, yes. I hadn't really considered that
> possibility, though there certainly is an argument for doing it that
> way. I should note that "Browse properties", too, shows up for pages
> that contain no properties - for those namespaces where SMW is enabled.
> Not that that's necessarily a reason for doing the same thing with Cargo.

The difference in SMW is that every semantics-enabled page has some 
property. :-)

Markus

>
> On Fri, Apr 3, 2015 at 4:22 PM, John McClure  > wrote:
>
> It would be nice if it only showed on pages for which it applies -
> is that possible?
>
>
> On 4/3/2015 1:20 PM, Yaron Koren wrote:
>>
>> Hi,
>>
>> Er... no? I didn't think it was a big deal; it's an exact analogue
>> of SMW's "browse properties" link.
>>
>> -Yaron
>>
>> Is it necessary that Cargo clutter all pages in a wiki with a
>> "page values" link?
>> --
>> thanks/john
>> skype:hypergrove
>>
>> 
>> --
>> Dive into the World of Parallel Programming The Go Parallel
>> Website, sponsored
>> by Intel and developed in partnership with Slashdot Media, is your
>> hub for all
>> things parallel software development, from weekly thought
>> leadership blogs to
>> news, videos, case studies, tutorials and more. Take a look and
>> join the
>> conversation now. http://goparallel.sourceforge.net/
>> ___
>> Semediawiki-devel mailing list
>> Semediawiki-devel@lists.sourceforge.net
>> 
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>
>
> --
> thanks/john
> skype:hypergrove
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
> --
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Sorting results

[Markus Krötzsch]

Hi Bernhard,

Why not just define a Number property for such pages and use this for 
sorting? If this does not work for you, then you could set the SORTKEY 
of the pages to something like 42 (with sufficiently many 0s to 
cover all numbers).

Cheers,

Markus

On 20.03.2015 15:47, Yaron Koren wrote:
> Hi Bernhard,
>
> Assuming that there's a standard naming system to this set of pages,
> i.e. it's always "[number] [other stuff]", one possibility is to have
> the template use #explode to get the number, then store that separately
> with a property of type "Number". Then you could sort on that property.
>
> -Yaron
>
> On Fri, Mar 20, 2015 at 4:57 AM, Krabina Bernhard  > wrote:
>
> Dear all,
>
> I understand that sorting results of a query depends on the type of
> property. So if you want to sort numbers, the property used for
> sorting has to be of type number, otherwise sorting will be
> alphabetical (1, 10, 2, 20, 3, 4,...)
>
> The problem is that if a property is of type page, SMW assumes that
> page name are alphabetical, and probably in 90% of the cases page
> names don't start with numbers. But there are cases, where I want
> numbers to be clickable...
>
> Could the order= parameter be enhanced to something like
> |order=numericasc
> |order=numericdesc
> to support numerical sorting of text/page-properties?
>
> regards,
> Bernhard
>
> 
> --
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your
> hub for all
> things parallel software development, from weekly thought leadership
> blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
> --
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] dumpRDF.php script in SMW 2.0 fails, problems with subobjects and redirects causes jobqueue to loop

[Markus Krötzsch]

Hi Daniel,

On 10.11.2014 20:31, Herzig, Daniel (AIFB) wrote:
>
> Problem #1: export of RDF fails with "Data for a subobject of PageA
> cannot be added to PageB'"

Yes, I encountered the same problem recently. The problem is the same 
when using Special:ExportRDF and possibly also when trying to use a 
SPARQL store (though I am not sure about this). It occurs whenever PageB 
redirects to PageA, and PageA has subobjects (including #ask queries).

The problem is based on SMW's handling of redirects. In query answering 
etc., SMW will follow redirects, i.e., when you use a redirect page in 
any request or query, you will get the answer as if you would have used 
the redirect target instead. There are some places where this can be 
switched off, but especially the SMWStore method getSemanticData() does 
not support this. This means that any attempt to get the semantic data 
(the stuff shown in the Factbox) for a redirect page will return the 
data for the redirect target.

You can see this effect when using Special:Browse on a redirect page. 
Notably, however, this does not cause any exception. Something else must 
be going wrong in the RDF export. My guess is that the call to 
getSubSemanticData() in SMW_ExportController.php(128) does in general 
cause an exception for SemanticData objects that have been created from 
redirects. Special:Browse would not run into this because the data for 
the subobjects is not expanded there (you only get links to show the 
subobject-data, but these links will trigger new requests that are not 
based on redirects).

The exception itself happens since a subobject of PageA is added to the 
data for PageB. This should not happen. In the situation with the 
redirect, the semantic data returned would need to use the redirect 
target as a subject if it wants to have the subobjects. This should be 
modified in the SQL store method. I am pretty confident that this will 
fix the exception.

This fix is not a complete solution for RDF, since it will mean that 
every redirect page exports the exact same data of its target page. So 
the dump will contain duplicate triples, and it will lack any 
information about redirects (they will not even be mentioned). A better 
handling would be to recognize that we are dealing with a redirect page, 
and to export it as such (in particular, do not call getSemanticData()). 
The right place to do this would be the method getSemanticData() in 
SMW_ExportController.php(294). It could check if a page is a redirect 
and then create a suitable SemanticData response manually for this case.

Another option would be to extend the store to support redirect 
resolution to be turned off for getSemanticData() by using some 
parameter. Care is needed there to make sure that the caching is not 
confused (but this seems easier if the subject is correctly converted to 
be the redirect target; so requests with and without redirect resolution 
will eventually use distinct subjects, and these should be the basis for 
caching).

Cheers,

Markus



> --
> When running SemanticMediaWiki/maintenance/dumpRDF.php following
> exception occurs, stack below [2]: 'Data for a subobject of PageA cannot
> be added to PageB'
>
> PageB is a redirect to PageA. PageB used lots of template with semantic
> data. When the information on PageB became outdated, a redirect has been
> inserted to point the newer page PageA. PageA uses also lots of
> templates with subobjects.
>
> Deleting page PageB solves the issue. However, there are thousands of
> redirects in this wiki and they can't be deleted.
> Unfortunately, I can not reproduce this issue in my test environment
> (same SMW and MW versions) and I have no developer access to the system
> having this problem.
>
> Did anybody encountered this issue too? Any ideas how to fix this?
>
> My assumption was that the reason for this bug lies in inconsistent
> semantic data. Maybe because some refreshJobs aren't done yet.
> When trying to debug redirects and data refresh jobs in my test
> environment, I encountered the problem with job loops and redirects,
> which might be connected:
>
>
> Problem #2: Infinite jobqueue runs
> --
> There was a discussion on infinite jobqueue runs recently on this list.
> However, I found this way easier to reproduce the problem.
> When inserting a redirect into a page that existed before and had
> semantic data on it, two SMW_UpdateJobs are inserted into the queue.
> When executed, these jobs spawn identical jobs, which causes the queue
> to loop forever.
> I can reproduce this behavior with the following procedure:
>
> * Create a page and insert a subobject. Save the page.
> * Edit the page again and insert a redirect, i.e. #REDIRECT
> [[AnotherPage]].  Save the page.
> ** The table 'smw_fpt_redi' has no entry for this page. I think there
> should be one, right?
> ** The subobject and its properties are 

Re: [SMW-devel] [Semediawiki-user] smw.referata.org vs. smw.org (Was: Exclude specific pages from query or #Concept?)

[Markus Krötzsch]

On 10.11.2014 17:59, James HK wrote:
> Hi,
>
>> Well, smw.referata.com is the community wiki and smw.org is the
>> official website of the extension. But maybe it is time to re-consider
>> if that separation is really that useful?
>
> I wasn't aware that such distinction actually exists and users are
> maintaining two sources. Unless there is (was) a real reason for such
> separation and for the sake of simplicity could we just use smw.org.

I think the separation was once made because smw.referata had some 
extensions than SMW.org (advantage: more features to showcase; 
disadvantage: problems upgrading to new SMW versions since some 
extensions were relying on specific SQL table layouts).

>
> I would like to see a working 2.0+ sandbox where people can try
> something out rather than having two wiki's with similar content.

Good point. We should fix the sandbox and have some way to auto-upgrade it.

Markus

>
> Cheers
>
> On 11/11/14, Stephan Gambke  wrote:
>> On 10 November 2014 17:22, pre...@gmail.com  wrote:
>>> I've updated the smw.referata.com page, but wonder why folks don't
>>> add/edit
>>
>> Thanks!
>>
>>> these tips into the semantic-mediawiki.org Wiki directly? I would have
>>> never
>>> found these tips on referata.com myself as the Main Page contains barely
>>> any
>>> content, such that unless one dug through the Categories or searched for
>>> key
>>> words directly, they'd never find these examples...
>>
>> Well, smw.referata.com is the community wiki and smw.org is the
>> official website of the extension. But maybe it is time to re-consider
>> if that separation is really that useful?
>>
>> --
>> ___
>> Semediawiki-user mailing list
>> semediawiki-u...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>>
>
> --
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] smw.org to provide a TemplateRepository (call for input)?

[Markus Krötzsch]

Hi James,

I like the idea, but maybe one should start with something that is more 
like a library of design patterns for SMW. Maybe you also had this in 
mind. There are endless amounts of types of content, and even person 
data could be specified in a wide range of ways, but there are maybe 
just a few typical patterns that are the same in most cases.

One could also consider a library of "content packages" that is similar 
to an extension library or a skin library. But this might need some more 
infrastructure (which format should packages and package descriptions be 
submitted in?). Then we could make this open and have contributed 
content packages that can be browsed and downloaded at smw.org.

In the latter case, it would be nice to encourage property/category 
reuse. If we have ten packages for managing persons, then it would be 
nice if they used the same properties for the same things.

Is there any option for internationalizing templates/forms in such a 
library?

Mobo looks very interesting. Maybe we could also encourage people to use 
this more, or even provide a platform where it could be used online. 
Results from this tool could then be published in our template library.

Cheers,

Markus


On 03.11.2014 01:44, James HK wrote:
> Hi,
>
> I'm calling for input and opinions about collecting and providing
> self-explanatory templates to be bundled as a repository (aka
> "TemplateRepository") on smw.org that are re-usable in connection with
> SMW.
>
> The objective would be to support first time users in their effort
> more quickly to become familiar with SMW by allowing to copy templates
> to enhance general functionality of Semantic MediaWiki right from the
> start.
>
> After reading the comment from [0] where sh(e) states that "I would
> like an inexperienced person of limited technical background to walk
> through some of the concepts with simple examples that I could cut,
> paste and modify in my own wiki. This would let me make mistakes
> because of what I don't know, but still move forward. "
>
> Making examples available for cut and paste usage seems a bit
> difficult (as it requires certain data sets to be available) but
> templates are a powerful tool in the "wiki world" to reduce input
> repetition and improve output presentation for specific content
> sections (info box etc.).
>
> Providing some kind of repository might come in handy for users
> unfamiliar with SMW and its inner workings where even an experienced
> user can be challenged to make templates work with SMW (depending on
> the requirements). Tools like [1] can provide support for a more
> complex modelling process but a first time or occasional user is
> probably looking for a simplified approach.
>
> Having a TemplateRepository where a user can browse (by use case),
> copy, and find some explanatory descriptions should ease the
> adaptation process and facilitate the usability of SMW.
>
> ## Questions
>
> - Would it make sense to have something like a TemplateRepository for
> re-use on smw.org?
> - How would a TemplateRepository need to be designed to be helpful to
> first time users or even expert users?
> - Do we have volunteers who are willing to help build a
> TemplateRepository? Do we have a champion who would be willing to
> drive such project?
> - What kind of TemplateRepository categories and common use cases
> (citation, book annotation etc.) would make most sense?
>
> ## Usage
>
> An easy way for users to download multiple templates would be by
> simply export a XML (e.g `Special:Export`) that contains all selected
> pages and use `Special:Import` to re-import the content without doing
> a manual copy and paste.
>
> Some categories to divide the usage could be based on an environmental
> dependency (can be used standalone, requires Lua, requires SF etc.).
>
> [0] http://www.semantic-mediawiki.org/wiki/20141102035706
>
> [1] https://github.com/Fannon/mobo
>
> PS: smw.org = https://www.semantic-mediawiki.org
>
> Cheers
>
> --
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] mStripState (Was: Ask query in extension)

[Markus Krötzsch]

Hi Frank, hi Mark, hi all,

The post below reminded me of how incredibly useful mStripState is when 
implementing parser hooks. It can be used to unstrip things (like 
) but also to insert things like  (e.g. to insert unsafe 
HTML from within a parser function).

It seems that there is no documented or recommended way to access this 
object in MediaWiki. Accessing a member field directly does not seem to 
be future safe. It would be nice if somebody could ask for a documented 
access to this, i.e., for "public function getStripState()", or we might 
all be up for some unpleasant surprises when somebody in MW gets around 
to declaring mStripState private ;-)

Cheers,

Markus


On 02.11.2014 12:12, Frank Baxmann wrote:
> Hi James,
>
> you can try this, works fine in similar case for me:
>
> 1) Create the ask string in php like this:
>
> /$search = "{{#ask: }}";/
>
> 2) Create the result string which you can manipulate later:
>
>  $resultString =
> $parser->mStripState->unstripBoth($parser->recursiveTagParse( $/search/,
> $frame ),$parser->mStripState);
>
> Greetings
>
> Frank
>
>
> On 01.11.2014 19:18, James Montalvo wrote:
>> I'm creating a parser function in which I'm going to include an ask
>> query. In this particular case I'd like it just return a normal HTML
>> table of results, as if I were to do:
>>
>> {{#ask: [[Category:Some category]]
>> |? Property 1
>> |? Property 2
>> | limit = 10
>> | sort = Property 2
>> }}
>>
>> Once I understand this I'll probably have cases where I need to modify
>> the results of the query before passing it to a result formatter or
>> formatting the results manually. What is the best way to do this using
>> existing SMW methods?
>>
>> Thanks,
>> James
>>
>>
>> --
>>
>>
>> ___
>> Semediawiki-devel mailing list
>> Semediawiki-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
> --
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] New (S)MW-based site that might be of interest

[Markus Krötzsch]

On 29.10.2014 19:12, Frank Baxmann wrote:
> Looks really nice.
> It would be interesting to be able to see the wikicode.

You can just add ?action=edit to any page URL. You can't make change, 
but you can see the code (be aware that the templates use a bit of a mix 
of German and English now, owing to the history of the data model). 
Special:Export is also freely available.

Cheers,

Markus

>
> On 29.10.2014 19:03, Markus Krötzsch wrote:
>> Hi all,
>>
>> I just finished work on a new website for my research group (and maybe
>> others). The site might be of interest, since it showcases quite a few
>> techniques in terms of skinning, customization, and SMW-usage:
>>
>> https://ddll.inf.tu-dresden.de/
>>
>> The main things to watch out for:
>>
>> * Chameleon skin with extensive use of Bootstrap layouts. I have written
>> several MW extensions to add more Bootstrap goodness to MW (esp. the
>> carousel at the start page, and the tabs you see in many places, but
>> also the affix second-from-top navigation bar that part-collapses on
>> small screens).
>> * Full-fledged dual-language MediaWiki. You can switch languages at the
>> top. I wrote quite a few hooks to make sure that pages switch
>> completely: navigation, main page, page contents, HTML headers. There is
>> still a site language (it's German) but the pages should be fully
>> localized (including an extra hreflang header link for Google).
>> * Complete deviation from MediaWiki's static navigation. Instead, all
>> side-column navigation (usually on the left of pages) is created by the
>> pages in the wiki themselves (using templates). This requires some small
>> extensions to allow me to move/hide/change the main title of the page
>> (you don't want the page title to span all columns and be above the
>> navigation menu). Using SMW, the navigation can be highly adaptive to
>> wiki and page contents. Examples:
>>
>> ** https://ddll.inf.tu-dresden.de/web/Ver%C3%B6ffentlichungen/2009/en
>> shows year numbers that come from an #ask query and puts the MediaWiki
>> __TOC__ for the current page below the currently selected year (with
>> some styling to make it fit visually).
>>
>> ** https://ddll.inf.tu-dresden.de/web/Article4007/en
>> shows links to publication pages of those authors (the authors come from
>> a template parameter added in a form) that have pages on the site
>> (again, SMW is used to figure out the latter).
>>
>> * There is a brand new Bibtex export that handles Unicode characters
>> (translating them to LaTeX commands) and uses the MediaWiki GeSHI
>> support for highlighting. See, e.g.,
>> https://ddll.inf.tu-dresden.de/web/Article4002/en Please contact me if
>> you are interested in this bibtex code; I have not published it anywhere
>> yet.
>>
>> The site is based on the original AIFB Portal [1] but with completely
>> reworked templates and skins (a lot of work I can tell you). The site is
>> of course not free for editing, but all relevant namespaces (including
>> Template and Form) are readable, and you can try action=formedit (do it
>> on German pages! the English ones are just querying data from SMW) to
>> see how the forms look.
>>
>> Cheers,
>>
>> Markus
>>
>> [1] http://www.aifb.kit.edu/web/Hauptseite
>>
>> --
>> ___
>> Semediawiki-devel mailing list
>> Semediawiki-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>
>
>
> --
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] New (S)MW-based site that might be of interest

[Markus Krötzsch]

Hi all,

I just finished work on a new website for my research group (and maybe 
others). The site might be of interest, since it showcases quite a few 
techniques in terms of skinning, customization, and SMW-usage:

https://ddll.inf.tu-dresden.de/

The main things to watch out for:

* Chameleon skin with extensive use of Bootstrap layouts. I have written 
several MW extensions to add more Bootstrap goodness to MW (esp. the 
carousel at the start page, and the tabs you see in many places, but 
also the affix second-from-top navigation bar that part-collapses on 
small screens).
* Full-fledged dual-language MediaWiki. You can switch languages at the 
top. I wrote quite a few hooks to make sure that pages switch 
completely: navigation, main page, page contents, HTML headers. There is 
still a site language (it's German) but the pages should be fully 
localized (including an extra hreflang header link for Google).
* Complete deviation from MediaWiki's static navigation. Instead, all 
side-column navigation (usually on the left of pages) is created by the 
pages in the wiki themselves (using templates). This requires some small 
extensions to allow me to move/hide/change the main title of the page 
(you don't want the page title to span all columns and be above the 
navigation menu). Using SMW, the navigation can be highly adaptive to 
wiki and page contents. Examples:

** https://ddll.inf.tu-dresden.de/web/Ver%C3%B6ffentlichungen/2009/en
shows year numbers that come from an #ask query and puts the MediaWiki 
__TOC__ for the current page below the currently selected year (with 
some styling to make it fit visually).

** https://ddll.inf.tu-dresden.de/web/Article4007/en
shows links to publication pages of those authors (the authors come from 
a template parameter added in a form) that have pages on the site 
(again, SMW is used to figure out the latter).

* There is a brand new Bibtex export that handles Unicode characters 
(translating them to LaTeX commands) and uses the MediaWiki GeSHI 
support for highlighting. See, e.g., 
https://ddll.inf.tu-dresden.de/web/Article4002/en Please contact me if 
you are interested in this bibtex code; I have not published it anywhere 
yet.

The site is based on the original AIFB Portal [1] but with completely 
reworked templates and skins (a lot of work I can tell you). The site is 
of course not free for editing, but all relevant namespaces (including 
Template and Form) are readable, and you can try action=formedit (do it 
on German pages! the English ones are just querying data from SMW) to 
see how the forms look.

Cheers,

Markus

[1] http://www.aifb.kit.edu/web/Hauptseite

--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] smw.org to replace the "old" Vector skin?

[Markus Krötzsch]

Hi,

On 21.10.2014 03:51, James HK wrote:
...
>
> PS: As far as I understand, the user of smw.org would be able to
> switch to the "old" Vector skin as registered user.
>

Let me come back to this, since this is also important for planning the 
transition. The key question is:

  Can we use Bootstrap components independent of Chameleon?

To make the in-page content fully responsive, we will need to use a grid 
framework like the one in Bootstrap. Many other nice components exist 
(tabs, carousels, etc.). If it is possible to use these with Vector as 
well as with Chameleon, then the transition is much easier and we could 
offer both skins alongside each other.

Markus


--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] smw.org to replace the "old" Vector skin?

[Markus Krötzsch]

On 20.10.2014 09:05, Krabina Bernhard wrote:
> Hi,
>
> regardless of the skin used, I think it's worthwile to have a look at some 
> sites we should aim to match regarding their appearance, e. g.
>
> http://www.drupal.org/
> https://wordpress.org/
> http://www.joomla.org/
> https://www.atlassian.com/

You are right. We really need to up our game there a little in terms of 
professional appearance. Looking at these very professional websites 
(and taking into account how much time it takes to do such a thing 
properly), we need to think about how to implement this. We do have a 
number of powerful components available (Chameleon, all the Bootstrap 
basics, I also have some not-yet-released extra code to use further 
Bootstrap features on the Wiki), but it will still be work to use that 
on our site (change table-based layouts on wiki pages; choose fonts, 
colours, images, and textures; style key elements; redesign site 
navigation to go with a collapsing menu, ...). How could we best move 
forward to put this into practice, ideally before the end of this year?

Cheers,

Markus


> - Am 17. Okt 2014 um 11:58 schrieb Stephan Gambke s7ep...@gmail.com:
>
>> Well, I plan to maintain the skin for a while yet, not least because I
>> use it myself.
>>
>> I want to finally release a version 1.0 this weekend. Should have done
>> that a long time ago, but always put it off. There is one major issue
>> left that I want to fix before, which is the un-responsiveness of the
>> navbar. Should not be too hard to do, somebody recently sent me some
>> code I think I can use. :)
>>
>> There are some rough edges left, but they are mostly related to
>> styling of special pages and the like and should be easy to fix
>> (patches welcome).
>>
>> Cheers,
>> Stephan
>>
>> On 17 October 2014 10:40, Markus Krötzsch  
>> wrote:
>>> Hi,
>>>
>>> Good idea. Chameleon is great (also because of the nice Bootstrap
>>> styling that becomes available on all pages). I am using it right now to
>>> build a site (to be published soon).
>>>
>>> A big advantage with Chameleon is that most customizations happen in CSS
>>> or by inserting HTML into the page through configuration files. so there
>>> are fewer dependencies on how MW does their skinning in PHP -- assuming
>>> that Chameleon is always updated to support this properly.
>>>
>>> My only concern would be that Chameleon is still classified Beta. It
>>> would be good to know if we can hope for future support for this if we
>>> switch now.
>>>
>>> Cheers,
>>>
>>> Markus
>>>
>>> On 16.10.2014 19:52, James HK wrote:
>>>> Hi,
>>>>
>>>> I'm calling for some opinions on replacing the "old" Vector skin on
>>>> smw.org with something fresh. Stephan in his relentless pursuit of
>>>> making MediaWiki more usable to non-Wikipedia users created the
>>>> Chameleon skin [0] which seems more suitable to those who want to use
>>>> MW/SMW in different environments.
>>>>
>>>> SMW no longer tries to be a product that seeks to fulfil requirements
>>>> from either WMF or Wikipedia to be used as encyclopaedic semantic
>>>> content management (that's now the job of Wikidata/Wikibase) therefore
>>>> using the "Vector" skin somehow appears to be outdated by trying to
>>>> look like the good old Wikipedia site.
>>>>
>>>> Furthermore, MW 1.24 (to be released soon) made some changes on how
>>>> skins are deployed [1] and it might be a good time to say farewell to
>>>> the "old" skin and rediscover SMW's identity by changing its
>>>> appearance.
>>>>
>>>> [0] https://www.mediawiki.org/wiki/Skin:Chameleon
>>>> [1]
>>>> https://www.mediawiki.org/wiki/MediaWiki_1.24#Skins_no_longer_loaded_after_upgrade.3F
>>>>
>>>> Cheers
>>>>
>>>> --
>>>> Comprehensive Server Monitoring with Site24x7.
>>>> Monitor 10 servers for $9/Month.
>>>> Get alerted through email, SMS, voice calls or mobile push notifications.
>>>> Take corrective actions from your mobile device.
>>>> http://p.sf.net/sfu/Zoho
>>>> ___
>>>> Semediawiki-user mailing list
>>>> semediawiki-u...@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>>>>


--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] smw.org to replace the "old" Vector skin?

[Markus Krötzsch]

Hi,

Good idea. Chameleon is great (also because of the nice Bootstrap 
styling that becomes available on all pages). I am using it right now to 
build a site (to be published soon).

A big advantage with Chameleon is that most customizations happen in CSS 
or by inserting HTML into the page through configuration files. so there 
are fewer dependencies on how MW does their skinning in PHP -- assuming 
that Chameleon is always updated to support this properly.

My only concern would be that Chameleon is still classified Beta. It 
would be good to know if we can hope for future support for this if we 
switch now.

Cheers,

Markus

On 16.10.2014 19:52, James HK wrote:
> Hi,
>
> I'm calling for some opinions on replacing the "old" Vector skin on
> smw.org with something fresh. Stephan in his relentless pursuit of
> making MediaWiki more usable to non-Wikipedia users created the
> Chameleon skin [0] which seems more suitable to those who want to use
> MW/SMW in different environments.
>
> SMW no longer tries to be a product that seeks to fulfil requirements
> from either WMF or Wikipedia to be used as encyclopaedic semantic
> content management (that's now the job of Wikidata/Wikibase) therefore
> using the "Vector" skin somehow appears to be outdated by trying to
> look like the good old Wikipedia site.
>
> Furthermore, MW 1.24 (to be released soon) made some changes on how
> skins are deployed [1] and it might be a good time to say farewell to
> the "old" skin and rediscover SMW's identity by changing its
> appearance.
>
> [0] https://www.mediawiki.org/wiki/Skin:Chameleon
> [1] 
> https://www.mediawiki.org/wiki/MediaWiki_1.24#Skins_no_longer_loaded_after_upgrade.3F
>
> Cheers
>
> --
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] MediaWiki Job queue problem

[Markus Krötzsch]

P.S. I forgot to mention that editing the redirect page again fixed some 
data in SMW (no more property values stored) but it still did not set 
the redirect marker set in smw_object_ids did not store the redirect in 
smw_fpt_redi. I did not test if this update might have stopped the 
cycle, but I doubt it (the earlier presence of spurious properties about 
illegal property values should not have affected the jobs). -- Markus

On 13.10.2014 11:29, Markus Krötzsch wrote:
> Hi again,
>
> I have isolated a single specimen of an infinite job cycle on my wiki.
> The details are attached (I hope the attachment makes it to the list as
> well). In short, the loop is triggered by an update job on a redirect
> page. For some reason, the update job creates another instance of a new
> update job of the same page.
>
> The SMW tables do not contain correct information about the redirect
> page: it has data stored about itself and is not marked as a redirect. I
> do not know if this is the cause of the problem or a side effect.
> However, since the page was stored with a #REDIRECT on it, this regular
> storing of the data should already have created correct data -- this
> should not depend on any update job.
>
> The file also contains a sample of a job queue that I had at first,
> where one of the indestructible jobs has two more copies of itself. They
> were never modified during my tests, but this might explain why a job
> queue can get longer and longer in such a case (new job instances,
> whereever they come from, are protected by their indestructible copies).
>
> The wrong SMW data might be the deeper issue here, but in any case it
> should be able to make the UpdateJob execution robust against this kind
> of cycle to address the main problem. Maybe the UpdateJob would (if
> successful) actually fix the data, though it can hardly be its cause.
>
> Regards,
>
> Markus
>
>
> On 25.09.2014 00:02, James HK wrote:
>> Hi,
>>
>>> I have executed runJobs several times and the job_attempts remains at
>>> 1 for
>>> those five jobs. We were thinking of doing a database backup today, then
>>
>> I'm curious about the "job_attempts" field as I would have expected to
>> see an increment for when the job (actually there has been an attempt
>> to execute and not only display a line on command shell) and to see
>> whether the job actually gets execute when running `runJobs`, just add
>> a simple `var_dump( 'hello world' )` line to [0] and verify a
>> `SMW\UpdateJob` activity.
>>
>> [0]
>> https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/includes/src/MediaWiki/Jobs/UpdateJob.php#L118
>>
>>
>> Cheers
>>
>> On 9/25/14, Daren Welsh  wrote:
>>> I have executed runJobs several times and the job_attempts remains at
>>> 1 for
>>> those five jobs. We were thinking of doing a database backup today, then
>>> delete those five jobs from the table, then run the SMW "repair and
>>> upgrade" via the admin special page.
>>>
>>> Even if this clears the job queue, we'd like to understand what
>>> caused this
>>> in the first place. I realize that's a very open-ended question :)
>>>
>>> Daren
>>>
>>>
>>> On Wed, Sep 24, 2014 at 4:30 PM, James HK 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>>> We currently have five jobs that are "stuck". All of them have 1 for
>>>>> job_attempts.
>>>>>
>>>>> One has job_cmd of refreshLinks in job namespace 10 and it is for a
>>>>> template page.
>>>>> The other four have job_cmd of SMW\UpdateJob in job namespace 0 and
>>>>> are
>>>> for
>>>>> "standard" pages. These pages do not seem to be related based on
>>>>> category
>>>>> or template.
>>>>
>>>> Just to make sure that I interpret the meaning of "stuck" correctly,
>>>> after finishing `runJobs` those four jobs (five with the
>>>> `refreshLinks` jobs) are still visible in the job table with an
>>>> "job_attempts" of 1. When running `runJobs` again the same four
>>>> `SMW\UpdateJob` (same as in the same title and same Id) jobs are
>>>> executed and increment the "job_attempts" to 2?
>>>>
>>>> If you empty the job table and execute `runJobs` does the same five
>>>> jobs appear again after the run with "job_attempts" = 1?
>>>>
>>>> Cheers
>>>>
>>>> On

Re: [SMW-devel] [Semediawiki-user] MediaWiki Job queue problem

[Markus Krötzsch]

Hi again,

I have isolated a single specimen of an infinite job cycle on my wiki. 
The details are attached (I hope the attachment makes it to the list as 
well). In short, the loop is triggered by an update job on a redirect 
page. For some reason, the update job creates another instance of a new 
update job of the same page.


The SMW tables do not contain correct information about the redirect 
page: it has data stored about itself and is not marked as a redirect. I 
do not know if this is the cause of the problem or a side effect. 
However, since the page was stored with a #REDIRECT on it, this regular 
storing of the data should already have created correct data -- this 
should not depend on any update job.


The file also contains a sample of a job queue that I had at first, 
where one of the indestructible jobs has two more copies of itself. They 
were never modified during my tests, but this might explain why a job 
queue can get longer and longer in such a case (new job instances, 
whereever they come from, are protected by their indestructible copies).


The wrong SMW data might be the deeper issue here, but in any case it 
should be able to make the UpdateJob execution robust against this kind 
of cycle to address the main problem. Maybe the UpdateJob would (if 
successful) actually fix the data, though it can hardly be its cause.


Regards,

Markus


On 25.09.2014 00:02, James HK wrote:

Hi,


I have executed runJobs several times and the job_attempts remains at 1 for
those five jobs. We were thinking of doing a database backup today, then


I'm curious about the "job_attempts" field as I would have expected to
see an increment for when the job (actually there has been an attempt
to execute and not only display a line on command shell) and to see
whether the job actually gets execute when running `runJobs`, just add
a simple `var_dump( 'hello world' )` line to [0] and verify a
`SMW\UpdateJob` activity.

[0] 
https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/includes/src/MediaWiki/Jobs/UpdateJob.php#L118

Cheers

On 9/25/14, Daren Welsh  wrote:

I have executed runJobs several times and the job_attempts remains at 1 for
those five jobs. We were thinking of doing a database backup today, then
delete those five jobs from the table, then run the SMW "repair and
upgrade" via the admin special page.

Even if this clears the job queue, we'd like to understand what caused this
in the first place. I realize that's a very open-ended question :)

Daren


On Wed, Sep 24, 2014 at 4:30 PM, James HK 
wrote:


Hi,


We currently have five jobs that are "stuck". All of them have 1 for
job_attempts.

One has job_cmd of refreshLinks in job namespace 10 and it is for a
template page.
The other four have job_cmd of SMW\UpdateJob in job namespace 0 and are

for

"standard" pages. These pages do not seem to be related based on
category
or template.


Just to make sure that I interpret the meaning of "stuck" correctly,
after finishing `runJobs` those four jobs (five with the
`refreshLinks` jobs) are still visible in the job table with an
"job_attempts" of 1. When running `runJobs` again the same four
`SMW\UpdateJob` (same as in the same title and same Id) jobs are
executed and increment the "job_attempts" to 2?

If you empty the job table and execute `runJobs` does the same five
jobs appear again after the run with "job_attempts" = 1?

Cheers

On 9/25/14, Daren Welsh  wrote:

We currently have five jobs that are "stuck". All of them have 1 for
job_attempts.

One has job_cmd of refreshLinks in job namespace 10 and it is for a
template page.
The other four have job_cmd of SMW\UpdateJob in job namespace 0 and are

for

"standard" pages. These pages do not seem to be related based on
category
or template.

On Wed, Sep 24, 2014 at 3:37 PM, James HK

wrote:


Hi,


runJobs.php will literally run forever. After the non-offending jobs
are
cleared it's easy to see which are the offenders. Thus far I think
all
offenders have been of type SMW::UpdateJob.


I don't think the problem is with the `SMW\UpdateJob` because it does
a simple "shallow update" of the store while the management of job
status (including how many attempts, id's etc.) are done by the MW
JobQueue (which has first change in 1.22 and then again in 1.23).

It does beg the question whether all `SMW\UpdateJob`'s are "stuck" or
only certain jobs belonging to a group of pages or single page?


runJobs.php, but for some reason they keep attempting to run over
and

over.

How do you know that the same job is run over and over again because
based and above discussion ("job_attempts") a job with too many
attempts is retired after some time.

If the same job is run over and over again, what is displayed for the
"job_attempts" counter?

[0] went into SMW 2.0 to counteract any possible job duplicates for
the same `root title`.

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/307

Cheers

On 9/25/14, James Montalvo  wrote:

I'm not sure if t

Re: [SMW-devel] [SF] Embedded Special:RunQuery creates strange charaters

[Markus Krötzsch]

Hi,

FWIW, these "strange characters" are placeholders that MediaWiki inserts 
into the text instead of "special" content (such as ,  and 
other tag-based functions). MW inserts these markers early on in the 
parsing process (in a phase called "stripping"), and most extension 
parser functions will only get to see the markers and not the original 
text. At the very end of parsing, MW will replace the markers with the 
desired content ("unstripping"). Normally, you get to see the markers 
when they have been stored accidentally in SMW. Another possible cause 
is that extensions are using their own parsers internally in an 
incomplete or buggy way (the markers left over by one parser instance 
cannot be unstripped by another parser instance -- they are only 
meaningful in the context of a single parsing process). This might occur 
during embedding.

I hope this can get you started (since you post to the developer list, I 
assume you want to dig into the code yourself to find out what's going 
on; debugging interactions of many extensions can require some original 
research).

Good luck,

Markus


On 08.10.2014 08:20, planetenxin wrote:
> Hi,
>
> embedded {{Special:RunQuery/Map_query}} creates strange characters like
>
> UNIQ225794f6d43c6688-h-0--QINU
>
> before headings or with {{#tree:}} parser function from
> Extension:TreeAndMenu. I'd guess that there are also other cases where
> this happens.
>
> Example: http://scratchpad.referata.com/wiki/Test_TreeAndQuery
>
> Any ideas?
>
> /Alexander
>


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] New extension for automatic page creation

[Markus Krötzsch]

Dear SMW users,

I have created a small extension, AutoCreatePage [1], that makes it 
possible to create "helper" pages by adding parser functions to one 
page. I think this is especailly useful for SMW (and SF) users, so I 
post it here.


How it works


You insert wikitext like the following into a page:

{{#createpageifnotex:Another page|Content for the page}}

When you safe the page, a new page "Another page" is created and its 
initial content is set to "Content for the page". This only happens if 
there is no page of that name yet. Pages are only created when saving an 
edit, not on preview or view. You can use  in the content part 
to insert templates, ask queries, and other "active" wikitext. The 
creation of the page is attributed to the user who made the edit. 
Created pages are prevented from creating further pages in the same edit 
(health and safety).


Why you would want this
===

When using SMW and Semantic Forms to create a pages, it can be useful to 
create several pages at once. For example, you could keep some long 
query results on their own page, or you could even create versions of 
the page in several languages (one is used as the master copy of all the 
data, and other language versions of the page use SMW to fetch information).

Normally, you would have to do this manually, even if the additional 
pages contain only templates and queries (which do not need any further 
user input to be created). With this extension, you can have templates 
(and forms) that create their own helper pages (and maybe link to them).

This is used heavily on the AIFB Portal [2]. My code is based on their 
original code, but most of it was rewritten to work with MediaWiki 1.23 
(in a cleaner way).


Credits
===

The original idea and code is due to Daniel Herzig. I just modernized 
and simplified the code.


Future
==

I just put this quickly on github to share it with others. If you like 
it, consider adopting the code (it is very short, only 100 lines). It 
would be good to move it to gerrit at some point to have 
internationalization of the three messages that it uses. Right now, the 
code should be considered in beta state, though I am not aware of any 
issues.


Cheers

Markus

[1] https://www.mediawiki.org/wiki/Extension:AutoCreatePage
[2] http://www.aifb.kit.edu/

--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW.org is down again

[Markus Krötzsch]

On 26/05/14 14:27, Michael Erdmann wrote:
> to whom it may concern.

Thanks, seems to happen whenever I am on a plane ;-) (and also the 
others who can fix it seem to have been busy). The problem is unusually 
frequent recently :-(

Markus

> --
>
> Dr. Michael Erdmann|erdm...@diqa-pm.com |   +49 151 6140 1790
> DIQA Projektmanagement GmbH | Pfinztalstr. 90 | 76227 Karlsruhe, Germany
> Handelsregister: Amtsgericht Mannheim HRB 715454 | USt-IdNr: DE283037270
> Geschäftsführer: Dr. Michael Erdmann,  Dipl.-Wirtsch.-Inf. Daniel Hansch
>
> This email may contain confidential information. If you are not the
> intended recipient please notify the sender immediately and delete this
> email. Any unauthorized copying, disclosure or distribution of this
> email is strictly forbidden.
>
>
> --
> The best possible search technologies are now affordable for all companies.
> Download your FREE open source Enterprise Search Engine today!
> Our experts will assist you in its installation for $59/mo, no commitment.
> Test it for FREE on our Cloud platform anytime!
> http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] semantic-mediawiki.org is down

[Markus Krötzsch]

On 07/05/14 13:18, Jamie Thingelstad wrote:
> You may want to consider signing up for a service like Uptime Robot
>
> http://uptimerobot.com
>
> I use it to monitor all of my sites and services. It’s free and does a great 
> job of letting you know when something is down.

Thanks for the hint. In fact, we already have a robot that alerts us. 
However, someone still has to restart the machine in this case, which 
can take a while depending on who is awake/at the keyboard. :-)

Markus

>
> On May 7, 2014, at 4:59 AM, Markus Krötzsch  
> wrote:
>
>> On 07/05/14 10:49, Krabina Bernhard wrote:
>>> Hi,
>>>
>>> semantic-mediawiki.org seens to be down...
>>
>> Rebootet right after your email, and running again since (I did this a
>> few times in recent days already; cause of problem remains unclear).
>>
>> Markus
>>
>>
>> --
>> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
>> • 3 signs your SCM is hindering your productivity
>> • Requirements for releasing software faster
>> • Expert tips and advice for migrating your SCM now
>> http://p.sf.net/sfu/perforce
>> ___
>> Semediawiki-devel mailing list
>> Semediawiki-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
> --
> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
> • 3 signs your SCM is hindering your productivity
> • Requirements for releasing software faster
> • Expert tips and advice for migrating your SCM now
> http://p.sf.net/sfu/perforce
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
• 3 signs your SCM is hindering your productivity
• Requirements for releasing software faster
• Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] semantic-mediawiki.org is down

[Markus Krötzsch]

On 07/05/14 10:49, Krabina Bernhard wrote:
> Hi,
>
> semantic-mediawiki.org seens to be down...

Rebootet right after your email, and running again since (I did this a 
few times in recent days already; cause of problem remains unclear).

Markus


--
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
• 3 signs your SCM is hindering your productivity
• Requirements for releasing software faster
• Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Wikimania 2014

[Markus Krötzsch]

Hi Bernhard,

Thanks for pushing this. I will prepare another Wikidata-related 
submssion and I am also happy to support any SMW-related events (I will 
certainly be in London).

Best regards,

Markus


On 31/03/14 09:29, Krabina Bernhard wrote:
> Hi,
>
> I just prepared a first draft of a submission: 
> https://wikimania2014.wikimedia.org/wiki/Submissions/Semantic_MediaWiki_as_OpenData_hub
>
> I appreciate your feedback on this and please sign at the bottom to support 
> the submission...
>
> regards,
> Bernhard
>
> - Ursprüngliche Mail -
>> Hi,
>>
>> does anybody plan to submit a SMW-related proposal to the Wikimania 2014?
>>
>> I do and I'm happy to team up with SMW evangelits ;-)
>>
>> Deadline is on Monday!  https://wikimania2014.wikimedia.org/wiki/Submissions
>>
>> regards,
>> Bernhard
>>
>> --
>> ___
>> Semediawiki-user mailing list
>> semediawiki-u...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>>
>
> --
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW 2.0 or 10.0?

[Markus Krötzsch]

Ok, I give in: +1 to "2.0"  :-)

Markus

On 18/01/14 19:27, Nischay Nahata wrote:
> I didn't follow the full discussion but 2.0 makes more sense to me.
>
>
> On Sat, Jan 18, 2014 at 11:15 PM, James HK  > wrote:
>
> Hi,
>
>  >From a visibility point of view, jumping from 1.9 to 10 seems a bit
> gigantic (even though you can argue the rational behind dropping 1
> where 10 follows 9) but I'm more in favour of going with 2 instead.
>
> Cheers
>
> On 1/19/14, Jeroen De Dauw  > wrote:
>  > Hey,
>  >
>  > It seems there is general agreement on having the next big SMW
> release have
>  > a semver.org  complaint version number.
>  >
>  > However there are two options that have been proposed and both
> work: 2.0
>  > and 10.0. The former one increments the first number, changing it
> from "the
>  > meaningless 1 at the front" to be the actual MAJOR version. The
> later drops
>  > the "meaningless 1", shifting the current actual MAJOR version
> "9" to its
>  > correct location, and then incrementing it as should be done with
> each new
>  > major release.
>  >
>  > I'm fine with both, though need to make some decision. So please
> indicate
>  > your preference.
>  >
>  > So far we have 1 point for 10.0 from Markus and 1 point for 2.0
> from Yaron.
>  >
>  > Cheers
>  >
>  > --
>  > Jeroen De Dauw
>  > http://www.bn2vs.com
>  > Don't panic. Don't be evil. ~=[,,_,,]:3
>  > --
>  >
>
> 
> --
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> 
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> Cheers,
>
> Nischay Nahata
>
>
> --
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Making SMW semver.org compliant

[Markus Krötzsch]

On 17/01/14 15:09, Yaron Koren wrote:
> Hi,
>
> Markus - are you suggesting retroactively referring to, say, version
> 1.5.1 as version 5.1 - with a corresponding change to all the
> documentation, etc.? Or just jumping straight from version 1.9 to
> version 10.0? I'm assuming the latter, but in either case that seems
> like a bad idea, that will in fact cause confusion, without any obvious
> benefit.
>
> Yes, Java did it, but, as Wikipedia notes [1], that was strictly for
> marketing purposes, presumably because "Java 6" looks a lot nicer on a
> book cover than "Java 1.6". Behind the scenes, Java developers still
> refer to it as 1.6, etc. When the first "SMW Unleashed" book comes out,
> then maybe it makes sense to start talking about making the number look
> nicer. :)
>
> I think it's overly dramatic to say that SMW versioning will never move
> past 1.x. There are a few previous versions that could have merited a
> jump to 2.0, in my opinion - the most recent two, 1.8 and 1.9, being
> obvious candidates. Perhaps it was a mistake to not make the major
> number jump with either of them. (And by the way, maybe it's not too
> late to rename 1.9 to 2.0? It's only been out for two weeks now...) But
> trying to pretend that those *were* major version number changes seems
> like it would cause more problems than it's worth.

I was suggesting to use 10 as the next version, but to keep all old 
releases (including 1.9) named as before; otherwise we would really 
cause confusion (e.g. there are archived emails about those versions 
that we cannot edit). However, the suggestion not to jump but to simply 
leave away the initial "1." in the future. My reason was that the number 
"9" seems to be our current major version, rather than a minor version 
of major version "1". And if it is good enough for Java marketing, I 
don't see a problem using it for SMW ;-)

Anyway, I won't fight for either option :-) Version 2.0 also sounds nice 
in a way.

Cheers,

Markus

>
> Jeroen - the numbering decisions of the developers of core MediaWiki
> seem irrelevant to this discussion, I would think.
>
> [1] https://en.wikipedia.org/wiki/Java_version_history
>
> -Yaron
>
>
> On Fri, Jan 17, 2014 at 8:59 AM, Jeroen De Dauw  > wrote:
>
> Hey,
>
>  > Markus:
>  > the next version should be 10.0.0
>
> Good point. +1 to this.
>
>  > Alexander:
>
>  > It is also important to tag all extensions. This is very often
> not the case yet...
>
> At least for SMW you can be sure all future releases will be tagged,
> since Composer essentially requires one to tag in order to create a
> release. The same is also true for SRF and Maps, and I hope more
> extensions will follow.
>
>  > Yaron:
>  > I can't imagine anyone would object to increasing the version
> number to 2.0, 3.0 etc. if/when it makes sense to do that.
>
> A discussion about the same question occurred quite some time back
> on wikitech. As you can probably guess, the outcome was rather
> different.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
>
> 
> --
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> 
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
> --
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
___

Re: [SMW-devel] Making SMW semver.org compliant

[Markus Krötzsch]

Hi,

+1 to dropping the initial "1." that will never ever change.

However, since the 1 was unchanged since we are out of beta, it seems to 
me that we are currently treating the second number as our major 
version. In other words, we are at version 9.0.1 and the next version 
should be 10.0.0. This is a bit like what was done for Java ;-).

I don't think users will be overly confused or upset in any case. Having 
more relevant first digits will certainly be good for clarity.

Cheers,

Markus

On 16/01/14 20:34, Jeroen De Dauw wrote:
> Hey,
>
> I think it would be nice if SMW was http://semver.org/ compliant.
>
> This means version numbers would look like MAJOR.MINOR.PATCH with a
> possible stability suffix. This is very close to what we are already
> doing, except that we are sticking a "1." in front of it. Having this
> shifted by one might not be confusing to people familiar with the SMW
> release cycle, though might be surprising to those who are not.
>
> How about switching to this schema for out next major release, which
> would then end up being 2.0?
>
> If we make this switch, the next major release seems like the best point
> to do this. Coincidentally one advantage to picking this release for
> such a change is that it makes us skip 1.10, which some users might
> think is smaller than 1.9.0.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
>
>
> --
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] OWA problem?

[Markus Krötzsch]

Hi John,

Apologies if you felt like getting a brick thrown through a window. 
Certainly not intended. My point was that formal terms from ontology 
languages cannot meaningfully be applied to software products, and the 
metaphor seemed a good way to illustrate the underlying idea.

Markus

P.S. I am not entitled to define what a CMS is or what class of software 
SMW is. For what I want to say, it only matters that it is a web 
application (of whatever type) and not an ontology language.

On 11/11/13 19:36, John McClure wrote:
> Markus, your stunning words below may be confusing to more than me. MW
> is a CMS, SMW is /not/. Discussing 'surface syntax' of MW's wikitext, is
> so irrelevant to SMW's semantic annotations of that wikitext, that I
> don't know where to even begin...
>
> If you'd rather not discuss the options I raised, that's perfectly fine!
> But throwing a brick through a window, is not.
> /jmc
>
> On 11/10/2013 11:42 PM, Markus Krötzsch wrote:
>> Hi John,
>>
>> The main point of my previous email was to explain that SMW does not
>> implement the CWA (or OWA). The terms CWA and OWA are used to
>> characterise knowledge representation formalisms. SMW, in contrast, is
>> a content management software. It has neither a formal syntax nor a
>> formal semantics. To say that "SMW implements the CWA" is similar to
>> saying that "SMW is green".
>>
>> Cheers,
>>
>> Markus
>>
>>
>> On 10/11/13 17:26, John McClure wrote:
>>> Hi Markus - thank you for your explanation of OWA -- for more, see also
>>> [1] [2].  My question is about the default for a property's type, so
>>> let's focus prescriptively on this.
>>>
>>> In OWL, two classes are used to define a property: ObjectProperty and
>>> DatatypeProperty; in RDF, just one. OWL permits us to sidestep this
>>> issue at hand because the range is indicated by the semantics of the
>>> relevant property class. RDF properties otoh do not force any
>>> committment about the range.
>>>
>>> Because there is a single "Property" wiki-namespace, SMW essentially
>>> implements , injecting >> rdf:resource='owl:ObjectProperty'/> when no type annotation is provided.
>>> Of course this is not equivalent to .
>>> In RDF,  without a range means that either a
>>> resource or literal can be the target of a triple whose predicate is
>>> Property:x.
>>>
>>> So I'm thinking of four options to clarify the situation - you may have
>>> others!
>>> 1) document the CWA that SMW implements
>>> 2) establish two property namespaces, vis a vis OWL
>>> 3) permit either text or page/subobject as targets
>>> 4) refuse to create a triple similar to datatype violations
>>>
>>> Just a few ideas kicking around -
>>> thanks
>>> [1] http://www.mkbergman.com/852/
>>> [2] http://www.cs.man.ac.uk/~drummond/presentations/OWA.pdf
>>> <http://www.cs.man.ac.uk/%7Edrummond/presentations/OWA.pdf>
>>>
>>> On 11/10/2013 2:33 AM, Markus Krötzsch wrote:
>>>> On 10/11/13 10:43, Niklas Laxström wrote:
>>>>> What is OWA?
>>>>
>>>> John refers to the "Open World Assumption". This is an informal
>>>> concept used in knowledge representation to describe the assumption
>>>> that statements that are not made are "unknown" rather than "false".
>>>> This is contrasted with the "Closed World Assumption" that is common
>>>> in databases, where we assume that unspecified information is false.
>>>>
>>>> For example, if the ACME company has a database that contains a table
>>>> for employees, and this table does not contain "Bob", then one would
>>>> assume that Bob is not an employee at ACME. In contrast, if an OWL
>>>> ontology contains information about people (e.g., using FOAF), and the
>>>> ontology does not contain Bob, one would not assume that Bob is not a
>>>> person (maybe Bob just has not created a FOAF file). If we want to say
>>>> that Bob is not a person in OWL, then we can do this directly by using
>>>> negation; in databases, this is usually not possible and we simply
>>>> assume that omitted information is negated.
>>>>
>>>> OWA is closely related to what we call "monotonicity": the more
>>>> information we enter into a system, the more informative it becomes.
>>>> Databases are not monoton

Re: [SMW-devel] OWA problem?

[Markus Krötzsch]

Hi John,

The main point of my previous email was to explain that SMW does not 
implement the CWA (or OWA). The terms CWA and OWA are used to 
characterise knowledge representation formalisms. SMW, in contrast, is a 
content management software. It has neither a formal syntax nor a formal 
semantics. To say that "SMW implements the CWA" is similar to saying 
that "SMW is green".

Cheers,

Markus


On 10/11/13 17:26, John McClure wrote:
> Hi Markus - thank you for your explanation of OWA -- for more, see also
> [1] [2].  My question is about the default for a property's type, so
> let's focus prescriptively on this.
>
> In OWL, two classes are used to define a property: ObjectProperty and
> DatatypeProperty; in RDF, just one. OWL permits us to sidestep this
> issue at hand because the range is indicated by the semantics of the
> relevant property class. RDF properties otoh do not force any
> committment about the range.
>
> Because there is a single "Property" wiki-namespace, SMW essentially
> implements , injecting  rdf:resource='owl:ObjectProperty'/> when no type annotation is provided.
> Of course this is not equivalent to .
> In RDF,  without a range means that either a
> resource or literal can be the target of a triple whose predicate is
> Property:x.
>
> So I'm thinking of four options to clarify the situation - you may have
> others!
> 1) document the CWA that SMW implements
> 2) establish two property namespaces, vis a vis OWL
> 3) permit either text or page/subobject as targets
> 4) refuse to create a triple similar to datatype violations
>
> Just a few ideas kicking around -
> thanks
> [1] http://www.mkbergman.com/852/
> [2] http://www.cs.man.ac.uk/~drummond/presentations/OWA.pdf
> <http://www.cs.man.ac.uk/%7Edrummond/presentations/OWA.pdf>
>
> On 11/10/2013 2:33 AM, Markus Krötzsch wrote:
>> On 10/11/13 10:43, Niklas Laxström wrote:
>>> What is OWA?
>>
>> John refers to the "Open World Assumption". This is an informal
>> concept used in knowledge representation to describe the assumption
>> that statements that are not made are "unknown" rather than "false".
>> This is contrasted with the "Closed World Assumption" that is common
>> in databases, where we assume that unspecified information is false.
>>
>> For example, if the ACME company has a database that contains a table
>> for employees, and this table does not contain "Bob", then one would
>> assume that Bob is not an employee at ACME. In contrast, if an OWL
>> ontology contains information about people (e.g., using FOAF), and the
>> ontology does not contain Bob, one would not assume that Bob is not a
>> person (maybe Bob just has not created a FOAF file). If we want to say
>> that Bob is not a person in OWL, then we can do this directly by using
>> negation; in databases, this is usually not possible and we simply
>> assume that omitted information is negated.
>>
>> OWA is closely related to what we call "monotonicity": the more
>> information we enter into a system, the more informative it becomes.
>> Databases are not monotonic in this sense: if I enter that Bob is an
>> employee, then I add some information (that "Bob is an employee") but
>> I also remove some information (that "Bob is not an employee"). OWL is
>> monotonic, but other knowledge representation languages are not.
>>
>> What John refers to is that SMW assumes a default type for properties
>> (Page). Therefore, the input behaviour is not monotonic: if you
>> specify another type later, then this will make the formerly true type
>> Page false. However, this is not a "violation" of the OWA. Wikitext is
>> just a surface syntax and not the internal knowledge model of SMW.
>> Wikitext can never be monotonic: for example, if a page contains
>> "[[someprop::somevalue]]" and you add an input character "X" to obtain
>> "[X[someprop::somevaue]]" then SMW will no longer contain the fact.
>> Parser functions, templates, comments, etc. will all lead to similar
>> effects.
>>
>> Clearly, it makes no sense for a surface syntax to be monotonic in
>> this sense: a meaningful notion of monotonicity cannot refer to the
>> character level. However, the only other structure that wikitext has
>> is the DOM tree of templates and parser functions. Basic SMW fact
>> syntax and MW category syntax does not feature there at all (it is
>> just character data). So it does not work for defining a meaningful
>> notion of monotonicity either. The simple conclu

Re: [SMW-devel] OWA problem?

[Markus Krötzsch]

On 10/11/13 13:06, Jonathan Lang wrote:
> On Nov 10, 2013, at 2:33 AM, Markus Krötzsch  
> wrote:
>
>> On 10/11/13 10:43, Niklas Laxström wrote:
>>> What is OWA?
>>
>> John refers to the "Open World Assumption". This is an informal concept
>> used in knowledge representation to describe the assumption that
>> statements that are not made are "unknown" rather than "false". This is
>> contrasted with the "Closed World Assumption" that is common in
>> databases, where we assume that unspecified information is false.
>
> -snip-
>
> Thank you for that; it was incredibly informative.  If you don’t mind, I have 
> a followup question of my own:
>
>> What John refers to is that SMW assumes a default type for properties
>> (Page). Therefore, the input behaviour is not monotonic: if you specify
>> another type later, then this will make the formerly true type Page
>> false. However, this is not a "violation" of the OWA. Wikitext is just a
>> surface syntax and not the internal knowledge model of SMW. Wikitext can
>> never be monotonic: for example, if a page contains
>> "[[someprop::somevalue]]" and you add an input character "X" to obtain
>> "[X[someprop::somevaue]]" then SMW will no longer contain the fact.
>> Parser functions, templates, comments, etc. will all lead to similar
>> effects.
>
> I think I understand what you’re saying here.  What I’d like to know is 
> whether any of this applies to the equivalency that exists (or is that past 
> tense?  I’m a bit behind the development curve right now) between OWL classes 
> and MW Categories?  That is, aren’t we making potentially unwarranted 
> assumptions about the semantic significance of a Category when we assume that 
> all Categories are classes, and vice versa?

That depends on how you use MW classes. Instances of OWL classes are 
propagated up the hierarchy: if A instanceOf B and B subclassOf C then A 
instanceOf C). MW classes do not behave like this. If one uses MW 
classes in a way for which this behaviour is not useful, then OWL 
classes are not a good way to capture this. In this case, the OWL export 
that SMW generates is of limited utility. Queries in SMW also interpret 
the category hierarchy in the same way as OWL, but this can be switched 
off if it does not work for you. Assuming that all MW categories in all 
wikis are intended to be used like OWL classes would certainly be an 
unwarranted assumption. SMW does not make that assumption, but if you 
install SMW on a wiki with its default configuration, then you choose 
this as your preferred way of treating categories.

Note that the previous paragraph speaks mainly about how a system 
behaves and whether this is desired or not. This is the only level on 
which we can discuss the relationship of MW and OWL -- considering them 
as pieces of technology that do something for the user. In particular, 
MW does not have a specified semantics (or even a formal syntax!), since 
it is not a format meant to exchange information across applications. So 
there is no right or wrong way of using MW categories, and one cannot 
say that they are generally compatible or generally incompatible with OWL.

Cheers,

Markus


--
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most 
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] OWA problem?

[Markus Krötzsch]

On 10/11/13 10:43, Niklas Laxström wrote:
> What is OWA?

John refers to the "Open World Assumption". This is an informal concept 
used in knowledge representation to describe the assumption that 
statements that are not made are "unknown" rather than "false". This is 
contrasted with the "Closed World Assumption" that is common in 
databases, where we assume that unspecified information is false.

For example, if the ACME company has a database that contains a table 
for employees, and this table does not contain "Bob", then one would 
assume that Bob is not an employee at ACME. In contrast, if an OWL 
ontology contains information about people (e.g., using FOAF), and the 
ontology does not contain Bob, one would not assume that Bob is not a 
person (maybe Bob just has not created a FOAF file). If we want to say 
that Bob is not a person in OWL, then we can do this directly by using 
negation; in databases, this is usually not possible and we simply 
assume that omitted information is negated.

OWA is closely related to what we call "monotonicity": the more 
information we enter into a system, the more informative it becomes. 
Databases are not monotonic in this sense: if I enter that Bob is an 
employee, then I add some information (that "Bob is an employee") but I 
also remove some information (that "Bob is not an employee"). OWL is 
monotonic, but other knowledge representation languages are not.

What John refers to is that SMW assumes a default type for properties 
(Page). Therefore, the input behaviour is not monotonic: if you specify 
another type later, then this will make the formerly true type Page 
false. However, this is not a "violation" of the OWA. Wikitext is just a 
surface syntax and not the internal knowledge model of SMW. Wikitext can 
never be monotonic: for example, if a page contains 
"[[someprop::somevalue]]" and you add an input character "X" to obtain 
"[X[someprop::somevaue]]" then SMW will no longer contain the fact. 
Parser functions, templates, comments, etc. will all lead to similar 
effects.

Clearly, it makes no sense for a surface syntax to be monotonic in this 
sense: a meaningful notion of monotonicity cannot refer to the character 
level. However, the only other structure that wikitext has is the DOM 
tree of templates and parser functions. Basic SMW fact syntax and MW 
category syntax does not feature there at all (it is just character 
data). So it does not work for defining a meaningful notion of 
monotonicity either. The simple conclusion is that wikitext is not a 
formal specification language, and that applying knowledge 
representation concepts like OWA and monotonicity to it is not meaningful.

If you want information specified in a formal knowledge representation 
format, then you can use the OWL/RDF exports of SMW. These exports have 
a monotonic semantics that is based on the OWA.

Markus



--
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most 
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] RFC: Technical documentation, installation instructions and release notes

[Markus Krötzsch]

Hi Jeroen,

different answers for different questions:

+1 to have detailed release notes in the source code (and nowhere else). 
The wiki will always have news items and SMW version pages that describe 
the main changes in a release; the release note file in the source can 
be linked from the wiki whenever more details are needed.


-1 to removing installation notes from the wiki. This is where many 
users are looking for them. When I did releases, I copied the 
installation instructions from the source code to a new wiki page, and 
this was almost no extra work (most work was writing these instructions 
in plain text first, since it is a bit cumbersome to do this without 
markup). The real problem that you describe is that the installation 
notes in the source code do not always get updated properly. This needs 
to be the responsibility of whoever does the release. It is hard to do 
this in a very agile way with many developers, for several reasons:

* a proper installation manual has a lot of text and some layout, which 
not every developer can be bothered with
* the instructions often depend on many interconnected changes, so it is 
often not so clear who is in charge to write a change (and it is easy to 
pretend that one is not)
* the instructions include things like version compatibility that are 
hard to fit into our own development process (for example, if MW 
releases a new version, then we need to say that we support it; but this 
is triggered by a change outside of our own code)
* installation notes need to include (released) version numbers of 
dependencies, while master could refer to git branches/revisions of 
other extensions instead

If you want, we could strip down the notes in git to a bare minimum and 
refer to a wiki page for the final details. This might make it easier 
for developers to keep the notes updated without getting into extnsive 
user documentation writing.


+0 to keep API documentation in the source. Most things are better 
documented in the source, the Web API is self-documenting, and the 
phpdocs should be better than wiki for code documentation. However, some 
general entry points and high-level introductions to these features and 
resources should be in the wiki, so people know where to go. However, 
hooks (=strings in some PHP function) do not appear in the source docs 
or API docs in a natural way; they might be better documented on the 
wiki, like MW does it.


Cheers,

Markus




On 05/11/13 09:57, Jeroen De Dauw wrote:
> Hey,
>
> We have been both shipping documentation together with the source code
> (as text files) and providing it on the SMW wiki since I joined the
> project. I see some problems with this.
>
> If the documentation is at both places, it doubles the maintenance
> effort. Given we have quite little resources to spend on this already,
> this does not help the quality of the documentation in question. Looking
> at the docs bundled with the source code, it seems that a lot of it is
> quite out of date, incomplete and sometimes plain wrong.
>
> A more focused effort seems to be in order. I propose that for the
> following items are only bundled with the source code and not duplicated
> elsewhere:
>
> * Release notes
> * Core installation instructions
> * Technical documentation on our APIs (and no, I' not talking just about
> our web API)
>
> If this is bundled with the source code, we can keep it fully up to date
> the whole time. If someone adds a new feature, they should also update
> the release notes. If someone changes an API, they should update the
> documentation. Relevant buzzword: "Agile documentation". Another
> advantage comes from the documentation not just being up to date with
> the latest development, it's also always correct for the version of the
> code you have. So if you get the previous release, you'll have the
> correct documentation for it, rather then something that reflects all
> the development that happened since. Yet another plus of having this
> together with the source is that people are not forced to hit our wiki.
> Offline use is possible. And it's just nice to have this in git together
> with the code, so no additional infrastructure is needed or relied upon.
>
> The SMW wiki would not contain a copy of this documentation. It can
> refer to it, and it can expand on it. Or provide translations. Technical
> tutorials, full installation guides and so on would also remain on the
> SMW wiki.
>
> What needs to happen to get there? We already made most of the change,
> just not explicitly. The biggest violators of this policy where the
> release notes, and our API documentation. The release notes for 1.8
> where actually already referenced from the SMW wiki at some point, by
> embedding the contents of the file from our git repo in the wiki page
> [0]. And we now have some documentation on our APIs in the "docs"
> folder, written in markdown format.
>
> Making such a change more explicit means that it is clear the
> documenta

Re: [SMW-devel] Moving SMW development to GitHub

[Markus Krötzsch]

Hi Jeroen,

making a writable secondary copy on GitHub is fine. It first sounded as 
it you wanted to abolish our gerrit repo completely (I think this is 
what people have replied to). We can also discuss this, but that will 
need a bit more time. The secondary GitHub repo will be a nive way to 
find out if this works well for us and for our (current and future) 
developers. So I am definitely in favour of trying this now (and thanks 
for clarifying the proposal).

Cheers

Markus

--
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Moving SMW development to GitHub

[Markus Krötzsch]

Hi Jeroen,

thanks for your proposal. It's very useful to revisit our infrastructure 
from time to time. Since people seem quite critical so far, we should 
probably have a bit more discussion

To be honest, I think that the (positive as well as negative) impact of 
this decision is not as big as one might believe. We have been at 
Sourceforge CSV initially, then moved to the MediaWiki SVN, then moved 
to MediaWiki git. None of this has really affected SMW development very 
deeply (although, to be fair, the move to MW SVN has increased 
contribution, but of couse we should not compare Sourceforge CSV to 
GitHub git when talking about contributions).

To me it seems that the coordination of two source control tools seems 
to be a lot of work. Niklas worried about the delay this could 
introduce, and Nischay worried (like others at SMWCon) about the 
confusion of having some extensions here and others their. Could we 
automate this somehow? If we had an automated synchronisation of GitHub 
and Gerrit, then the advantages would combine and the disadvantages 
would vanish, right?

Cheers,

Markus


On 02/11/13 16:20, Jeroen De Dauw wrote:
> Hey,
>
> Thanks all for your feedback.
>
> Since there is apparently a good degree of apprehension on changing this
> workflow, which is not all to surprising, I'll slightly alter and
> reframe the proposal.
>
> Rather then "switching" to GitHub, we'll just have the new repo on
> GitHub via which we access pull requests. The workflow on the WMF hosted
> repo does not change. If things go well, we can gradually move more
> activity to GitHub.
>
> Below I'm addressing some raised questions and concerns:
>
>  > Are there similar plans for the SMW-related extensions?
>
> It is really up to the maintainers of those extensions. I'm quite
> unhappy with the one-size-fits-all attitude that is quite prevalent in
> the MediaWiki community. In some cases it makes sense to provide support
> via GitHub, in others only accepting contributions via Gerrit is the
> better approach. I personally have no concrete plans for extensions such
> as Semantic Maps at present. If SMW benefits from this change, then I'll
> of course be considering it. For the Wikidata project, we are moving
> several PHP libraries to GitHub, some of which are SMW related.
>
>  > Wouldn't it be confusing if some work is on Gerrit (MW and many other
> extensions) while SMW work is on Github?
>
> The number of potential SMW developers that are using GitHub is
> significantly greater then the number of MediaWiki developers. And as we
> have clear empirical evidence by now, the later group is not very eager
> to contribute to SMW, so why should we throw the former, under the bus?
> Most interest in SMW is coming from outside the core MediaWiki
> community, and for them the current workflow is not only confusing, it
> is a big hassle.
>
>  > Also worth noting would be Yuvi's Github to Gerrit Bot
> http://www.mediawiki.org/wiki/User:Yuvipanda/G2G
>
> Relying on this mirroring infrastructure strikes me as added complexity
> and more maintenance hassle. The fewer infrastructure that can break in
> such a way it seriously obstructs the development process, the better.
>
>  > Will it speed up the review process in some way? :-)))
>
> Not initially. However if we get more people contributing, this
> situation could improve. That is all very speculative however.
>
>  > This sounds like a potential for a mess, merging stuff into both
> directions.
>
> Git is a distributed version control system. Distributed. In the end
> there is little difference between having a branch on a repo hosted on
> GitHub or a branch on my local box. The kind of merging that will happen
> will not differ much from what already happens. The tree might get
> deeper and more complex if people start creating forks and whatnot,
> though again, this is what Git is made for.
>
> I anticipate no hassle whatsoever for i18n updates, as they will still
> come from one source, which is the same as our current situation.
>
>  > From my point of view, you would lose visibility of development
>
> Ah, this is part of the promise that everything will be so much better
> if one uses a WMF hosted git repo. As I already implied above, this
> promise fails to deliver. This is the case for SMW, and for most
> extensions. There might be exceptions, though that does not help us.
>
> pre-testing (some of the extensions) at translatewiki.net
>  as well as
> having new releases with fewer translations, because translators have
> less or no time to translate new strings. With this scheme it can take
> two releases to fix an i18n problem and having a good translation for
> it (fix in next release, translations in next+1 release).
>
>
> Doing a push from one repo to another is not very difficult, so we can
> frequently update the WMF copy of the code, and not lose the TWN
> testing. Also note how this proposal does not include extensi

[SMW-devel] Fwd: Wikidata Toolkit: call for feedback/support

[Markus Krötzsch]

Dear all,

FYI: a project proposal for creating a software toolkit to work with 
Wikidata data in external tools [1]. Sent to wikidata-l, but maybe it's 
also interesting for those working with semantic data here.

Feedback and support is welcome. Having email replies go to wikidata-l 
would be great, but don't let this hold you back if you're not on there. 
You can also post to the bottom of the wiki page [1].

Cheers,

Markus

[1] https://meta.wikimedia.org/wiki/Grants:IEG/Wikidata_Toolkit


 Original Message 
Subject: [Wikidata-l] Wikidata Toolkit: call for feedback/support
Date: Sun, 29 Sep 2013 13:11:11 +0100
From: Markus Krötzsch 
Reply-To: Discussion list for the Wikidata project. 

To: Discussion list for the Wikidata project. 


Dear Wikidatanions (*),

I have just drafted a little proposal for creating more tools for
external people to work with Wikidata, especially to build services on
top of its data [1]. Your feedback and support is needed.

Idea: Currently, this is quite hard for people, since we only have WDA
for reading/analysing dumps [2] and Wikidata Query as a single web
service to ask queries [3]. We should have more support for programmers
who want to load, query, analyse, and otherwise use the data. The
proposal is to start such a toolkit to enable more work with the data.

The plan is to kickstart this project with a small team using
Wikimedia's Individual Engagement program. For this we will need your
support -- feel free to add your voice to the wiki page [1]. Of course,
comments of all sorts are also great -- this email thread will be linked
from the page. If you would like to be involved with the project, that's
great too; let me know and I can add you to the proposal.

The proposal will already be submitted tomorrow, but support should also
be possible after that, I hope.

Cheers,

Markus


(*) Do we have a demonym yet? Wikipedian sounds natural, Wikidatan less
so. Maybe this should be another thread ... ;-)

[1] https://meta.wikimedia.org/wiki/Grants:IEG/Wikidata_Toolkit
[2] http://github.com/mkroetzsch/wda
[3] http://208.80.153.172/wdq/

-- 
Markus Kroetzsch, Departmental Lecturer
Department of Computer Science, University of Oxford
Room 306, Parks Road, OX1 3QD Oxford, United Kingdom
+44 (0)1865 283529   http://korrekt.org/

___
Wikidata-l mailing list
wikidat...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-l



--
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] No response for inline query with RDF store

[Markus Krötzsch]

Hi Kai,

On 10/09/13 19:22, Hüner Kai wrote:
> Dear SMW developers,
>
> I already posted this question to the SMW user group, but maybe it is more 
> development related.
> Sorry for double posting, but I really have no idea how to proceed.
>
> I want to use SMW (MW: 1.21.1, SMW: 1.8.0.5) together with a Sesame RDF 
> database.
> Everything works fine, data from SMW is updated in the RDF database, and 
> SPARQL queries (e.g. from Java API) are answered correctly.
>
> Exception: Inline queries in the wiki itself, if categories are used!
> (1) '{{#ask: [[Germany]] | ?Has official name }}' works fine (obvious test 
> scenario: Set of countries).
> (2) But '{{#ask: [[Category:Country]] | ?Has official name }}' does not 
> provide any result.
>
> If I disable the RDF store (i.e. comment  $smwgDefaultStore = 
> 'SMWSparqlStore'; in LocalSettings.php), both inline queries work absolutely 
> fine. If I enable the RDF store again: No results for (2).
> Of course, I run SMW_refreshData.php -ftpv and -v multiple times.
>
> If there is no "easy" reason for this problem, can anybody recommend another 
> RDF store? I just want to have a simple SPARQL endpoint for SMW with a Java 
> API.

In general, there are two possible intermediate causes why a query might 
fail to return results: (1) the data was not stored correctly in the RDF 
store or (2) the query is not understood correctly by the RDF store 
(which might also be due to the query not being generated correctly by SMW).

To debug, please run your query with "format=debug". This will give you 
the SPARQL query that SMW is using. You can then ask this query against 
your store directly to see if you get any results. If not, try to find 
out if there is data missing in the store (by running modified queries 
that show the stored data) or if the query is somehow wrong (e.g., using 
another URI when querying Categories than when storing them).

One more thing: SMW in SQL supports a category hierarchy like a class 
hierarchy. When searching for things in Category:Country, you will also 
find all things that are in subcategories of country (say 
Category:EU_Country). This is done by SMW for SQL. For RDF, SMW relies 
on the RDF's store ability to interpret subclassOf -- if it does not do 
this, then subcategories will be ignored.

Cheers,

Markus



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW + Selenium

[Markus Krötzsch]

On 21/09/13 00:06, Jeroen De Dauw wrote:
> Hey,
>
> If these tests are broken and no one is maintaining them, I suggest we
> remove them from master. They can be resurrected from version history if
> anyone cares to fix them.
>

+1 if nobody uses them right now (I don't)

Markus


--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] RFC: SMWPageSchemas in SMW core (1.9)

[Markus Krötzsch]

On 08/08/13 13:18, Jeroen De Dauw wrote:
> Hey,
>
> I concur with James on the points he made.
>
> The approach taken would make more sense if SMW was a plug in to PS, as
> this is what the code makes it out to be. I'd rather see the code be
> part of PS, and have it only be registered there is SMW is loaded.

+1

Markus

>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
>
>
> --
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Coveralls.io support

[Markus Krötzsch]

Great, thanks for setting this up.

Markus

On 07/08/13 08:36, Jeroen De Dauw wrote:
> Hey,
>
> Our TravisCI builds [0] now run code coverage and report the result to
> coveralls.io  [1]. This makes it easy for anyone to
> view the current coverage level, and to keep track of evolution over
> time. You can even see coverage changes per file per commit.
>
> [0]
> https://travis-ci.org/wikimedia/mediawiki-extensions-SemanticMediaWiki/builds
> [1] https://coveralls.io/r/wikimedia/mediawiki-extensions-SemanticMediaWiki
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
>
>
> --
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Showing triples in SMW externally added via SPARQL

[Markus Krötzsch]

Hi Michael,

what you would like to do is probably not possible in SMW right now, but 
it might be possible to get it working. When using an RDF store, SMW 
stores (copies of) its data there and queries the store to obtain result 
lists for #ask. However, all simple data lookups are still done through 
the SQL database backend. So, for example, if you have a table generated 
by #ask, then SPARQL will be used to find out which results are in the 
table (one result = one row). But the contents of the table cells 
(property values) still come from the database.

There have been extensions to SMW that allow you to SPARQL query 
(directly) external data sources, but that was not integrated with wiki 
data.

In principle, it would not be too hard to re-wire all of SMW's data 
access to the RDF store if desired. Some development is needed to do 
this, but I think only in two or three files. There are two main tasks:

* The individual data access methods of the SMWSparqlStore 
implementation would have to be implemented to run and return results 
from SPARQL queries. Currently this code just inherits from the SQL 
store and overwrites very little of its public interface (see SMWStore 
for documentation of the available methods). For getting SPARQL to 
access the RDF store, it should be enough to implement the methods that 
returns all data about one subject (getSemanticData). The method that 
gets data values for one subject and property largely is based on 
getSemanticData. Overall, this should not be too hard, but it needs to 
be done. The SQL store code is more complicated, since it needs to fetch 
data from many tables and since it uses caching (though you might want 
to use that too) -- SPARQL might actually be comparatively simple.

* SMW's SPARQL binding/RDF mapping code must be extended to be able to 
read values from SPARQL results that have datatypes other than Wikipage. 
Currently, there is no code to read, say, a number that is returned by a 
SPARQL query (but there is code to parse SPARQL query results in 
general; just the specific value parsing is missing). One could also 
defer this task and start with wikipage property data only to see how it 
goes.

Another challenge in the SPARQL result parsing might be that the SPARQL 
result interpretation in SMW is not really designed to process result 
tables of the form "Property|Value" where value could contain literals 
of any type. Maybe one needs another access method in the SPARQL code to 
turn this into a SemanticData object. Note that you also need to make 
sure that the SMW datavalue ends up in a single RDF value (e.g., 
coordinates have latitude and longitude, but there should also be a 
string encoding that combines both for retrieval). If a datavalue is 
spread across multiple triples, then it would be much harder to retrieve 
it. I would rather add a value to SMW's RDF export for convenient access 
than to write a more complicated access method.

One thing to be checked (I don't recall) is the use of graph names in 
SMW. I think SMW stores all of its data in one graph, so it would be 
unproblematic to store more data with the same URIs as long as it's in a 
different graph. The question then is how normal SPARQL queries use the 
graph setting (otherwise you might find that your new data is not shown 
even after extending methods as required).

I guess it is clear, but what you will of course never be able to do is 
to modify RDF data that was written by SMW in the RDF store. The data 
that SMW stores is taken from wiki pages and changing the data in the 
RDF (or in SQL) won't change the pages. So the next time somebody edits 
a page (or a refresh job happens), the old data would overwrite the new.

Cheers,

Markus


On 06/08/13 14:59, Michael Färber wrote:
> Hi all,
>
> I have SMW with a triple store (Virtuoso) connection and want to store
> additional structured data into this triple store by an external tool
> (by accessing the triple store access point directly with SPARQL, i.e. I
> add triples externally). Is there a viable way to get this added data
> (properties) shown in the SMW frontend/pages? Whenever I insert triples
> in the triple store, SMW ignore these, although I took care of the URI
> naming conventions.
> It seems that SMW also considers the triples in the MySQL DB. Can't I
> just determine that SMW should only consider the triplestore structured
> data?
> I have heard of using a separate graph in the triple store for storing
> and quering triples I added by myself externally, but using only one
> graph would make life much easier in my opinion, since I use the same
> properties.
>
> Thank you very much for your support.
>
> Best,
> Michael
>
>
> --
> Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version co

Re: [SMW-devel] [Semediawiki-user] Replacement for Wiki of the Month Coordinator

[Markus Krötzsch]

On 21/06/13 13:46, Yury Katkov wrote:
> Is that possible to create a working group for defining Wiki of the
> Month or do that collaboratively? Some people would want to judge but
> they don't have enough disipline to do that every month.

In any case we need a person who is in charge -- many people without 
discipline do not replace one person with ;-). If two people want to 
share the task and be in charge every other month, this would also work, 
but it should be clear for each month who is making sure that there is a 
WOTM.

I suggest to first find the new coordinator(s), and then let 
her/him/them consider new voting approaches if they think it makes sense.

Markus

>
>
> On Tue, Jun 4, 2013 at 11:24 AM, Markus Krötzsch
> mailto:mar...@semantic-mediawiki.org>>
> wrote:
>
> Hi Desiree,
>
> first to the second part of your message: this is really great news,
> congratulations!
>
> The not-so-great news (though it hardly matches the other) is that we
> now need somebody to step up as WOTM coordinator. Before we worry about
> that, let me first thank you for your work -- you did a truly wonderful
> job on this. WOTM makes a great contribution to SMW, both as a resource
> for people to learn about SMW and for projects to showcase their sites.
>
> When we started this feature in 2010, we were not sure if this would
> continue for more than a year -- already back then, it has not always
> been easy to find suitable candidates for each month. You managed to
> keep the series running, and I was often impressed by the variety of
> great sites you (and Yaron) came up with. By today, WOTM is not just a
> catalogue of SMW sites but a showcase of original and inspiring
> community projects from around the world. I really hope that we can find
> somebody to continue this success story.
>
> (To those who consider becoming volunteers: just ask us if you are not
> sure what this job requires.)
>
> So thanks again, Desiree, and all the best wishes to the three of you,
>
> Markus
>
>
>
> On 03/06/13 19:14, Desiree Gennaro wrote:
>  > Hello Everyone,
>  >
>  > Unfortunately, I am going to need to find a new volunteer to take
> over
>  > coordinating the featured wiki of the month starting in August.
> My husband
>  > and I will be welcoming the birth of our first child, Carly, in early
>  > September. I know myself and Yaron will be available to answer any
>  > questions and provide guidance as needed to make the transition
> go more
>  > smoothly.
>  >
>  > Anyone who is interested can send an email to
>  > wikiofthemo...@semantic-mediawiki.org
> <mailto:wikiofthemo...@semantic-mediawiki.org> and I will gladly
> answer any
>  > questions or provide any additional information requested.
>  >
>  > Best Regards,
>  > Desiree Gennaro
>  >
> 
> --
>  > How ServiceNow helps IT people transform IT departments:
>  > 1. A cloud service to automate IT design, transition and operations
>  > 2. Dashboards that offer high-level views of enterprise services
>  > 3. A single system of record for all IT processes
>  > http://p.sf.net/sfu/servicenow-d2d-j
>  > ___
>  > Semediawiki-user mailing list
>  > semediawiki-u...@lists.sourceforge.net
> <mailto:semediawiki-u...@lists.sourceforge.net>
>  > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>  >
>
>
> 
> --
> How ServiceNow helps IT people transform IT departments:
> 1. A cloud service to automate IT design, transition and operations
> 2. Dashboards that offer high-level views of enterprise services
> 3. A single system of record for all IT processes
> http://p.sf.net/sfu/servicenow-d2d-j
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> <mailto:semediawiki-u...@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>
>


--
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Replacement for Wiki of the Month Coordinator

[Markus Krötzsch]

Hi Desiree,

first to the second part of your message: this is really great news, 
congratulations!

The not-so-great news (though it hardly matches the other) is that we 
now need somebody to step up as WOTM coordinator. Before we worry about 
that, let me first thank you for your work -- you did a truly wonderful 
job on this. WOTM makes a great contribution to SMW, both as a resource 
for people to learn about SMW and for projects to showcase their sites.

When we started this feature in 2010, we were not sure if this would 
continue for more than a year -- already back then, it has not always 
been easy to find suitable candidates for each month. You managed to 
keep the series running, and I was often impressed by the variety of 
great sites you (and Yaron) came up with. By today, WOTM is not just a 
catalogue of SMW sites but a showcase of original and inspiring 
community projects from around the world. I really hope that we can find 
somebody to continue this success story.

(To those who consider becoming volunteers: just ask us if you are not 
sure what this job requires.)

So thanks again, Desiree, and all the best wishes to the three of you,

Markus



On 03/06/13 19:14, Desiree Gennaro wrote:
> Hello Everyone,
>
> Unfortunately, I am going to need to find a new volunteer to take over
> coordinating the featured wiki of the month starting in August. My husband
> and I will be welcoming the birth of our first child, Carly, in early
> September. I know myself and Yaron will be available to answer any
> questions and provide guidance as needed to make the transition go more
> smoothly.
>
> Anyone who is interested can send an email to
> wikiofthemo...@semantic-mediawiki.org and I will gladly answer any
> questions or provide any additional information requested.
>
> Best Regards,
> Desiree Gennaro
> --
> How ServiceNow helps IT people transform IT departments:
> 1. A cloud service to automate IT design, transition and operations
> 2. Dashboards that offer high-level views of enterprise services
> 3. A single system of record for all IT processes
> http://p.sf.net/sfu/servicenow-d2d-j
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] URIResolver's urly URLs in 4store

[Markus Krötzsch]

On 11/05/13 20:35, Yury Katkov wrote:
> When I use 4store as database for properties my cyrillic URLs are very
> urly. Here is one of examples:
>
> http://127.0.0.1/wikivote/index.php/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F:URIResolver/-D0-A1-D0-B2-D0-BE-D0-B9-D1-81-D1-82-D0-B2-D0-BE-3A-D0-94-D0-B0-D1-82-D0-B0_-D0-BF-D0-BE-D1-81-D0-BB-D0-B5-D0-B4-D0-BD-D0-B5-D0-B9_-D0-BF-D1-80-D0-B0-D0-B2-D0-BA-D0-B8-23aux
>
> Isn't that an awful URL? I would be much happier if I had something
> like that instead:
> http://127.0.0.1/wikivote/index.php/Служебная:URIResolver/Заглавная_страница
>
> Is that possible?

The URLs currently go through an encoding that eliminates all non-ascii 
characters (URL encoding) and then replaces % by -. This ensures that 
all generated URLs also can be used in MediaWiki links (e.g., in article 
names pointing to the Special URI resolver page), and that we can always 
produce valid RDF/XML exports (this syntax requires some URI parts to 
occur in XML element names, which leads to additional constraints). I 
still think that one would not need to escape Cyrillic letters for these 
reasons. If you have code that does the job better (encoding and 
decoding, capturing all symbols that cause trouble in XML or MW but 
nothing else), then we could make this an option (all of this encoding 
is in one file, so it can be changed).

Markus

> -
> Yury Katkov, WikiVote
>
> --
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and
> their applications. This 200-page book is written by three acclaimed
> leaders in the field. The early access version is available now.
> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Any chance to get special properties to the factbox?

[Markus Krötzsch]

Dear Steve,

I think it is just a matter of finding the right hook here. It should 
not be related to a new type of edit action, but simply some hook that 
is triggered whenever MediaWiki parses a page. For a list of hooks that 
run during parsing, see:

http://www.mediawiki.org/wiki/Category:MediaWiki_hooks_included_in_Parser.php

You could use many of these, since you do not need to look at the text 
(so it does not matter for you at which stage of processing the parser 
is). SMW processes [[property::value]] annotations during 
InternalParseBeforeLinks.

Cheers,

Markus



On 24/02/13 07:51, Lists wrote:
> Hallo Markus,
>
> If I use a custom MW-parser function hook like this:
>
> [...]
>
> $action = MediaWiki::getAction();
>
>  if($action == 'formedit'):
>
>$dataItem = new SMWDIBlob('Das ist ein zweiter Test');
>$property = new SMWDIProperty('___SDWTEST');
>if(!is_null($dataItem)):
>   SMWParseData::getSMWData($parser)->addPropertyObjectValue($property,
> $dataItem);
>endif;
>
>   endif;
>
> ...it's set the property but it is still only visible in the browse mode of
> the factbox.
> If I then trigger SMW_refreshData.php it erases the complete property for
> the page even
> I use a commandline exception like this:
>
> if($wgCommandLineMode==true):
>  return true;
>  endif;
>
> So this means there is no way to inject an edit-action with automated
> property data unless you use
> special properties and 'SMWStore::updateDataBefore'. Is this right? We have
> tried hidden fields in SF too.
> But the hidden field-implementation of Semantic Forms is some kind buggy. We
> also modified the markup at runtime which
> results in ugly tagging code for the user.
>
> Are there any functional alternatives to inject an edit-action to provide
> automated semantic data values, which will stay after
> triggering SMW_refreshData.php?
>
> c u steve
>
>
> -Ursprüngliche Nachricht-
> Von: Markus Krötzsch [mailto:mar...@semantic-mediawiki.org]
> Gesendet: Samstag, 23. Februar 2013 19:14
> An: Lists
> Cc: semediawiki-devel@lists.sourceforge.net
> Betreff: Re: [SMW-devel] Any chance to get special properties to the
> factbox?
>
> On 23/02/13 17:36, Lists wrote:
>> Hello Markus and the others,
>>
>>
>> we are doing this in a "SMWStore::updateDataBefore" - handler. The
>> property shows up in the browse function not in the "normal" article
>> view. If this isn't functional how can we alter an existing property
>> value with this handler?
>> Let's say we have a property called "Edited by". This property should
>> have the value (string) "Hans Bot" which would be set programatically
>> in an "SMWStore::updateDataBefore - handler" while saving an article.
>> How to archieve this?
>
> Properties that should appear in the Factbox need to be added during
> parsing, so you would have to use another MediaWiki hook that is triggered
> during parsing. The SMWStore hook is called only when the page is saved, so
> it does not happen at all on preview, so it cannot have any effect on the
> Factbox. There are many MW hooks that run during parsing, but we do not have
> a standard pattern or example code for hacking into the system at this
> point, I think. You can have a look at the SMW code if you like
> (SMWParseData implements the main methods used to store data during parsing;
> most importantly, you can get the SMWSemanticData object during parsing by
> calling SMWParseData::getSMWData( $parser ); if you store data in there
> during parsing then it should appear in the Factbox).
>
> Cheers,
>
> Markus
>
>>
>> For now we have this for a special/meta property which doesn't fits:
>>
>> //Edited by
>>   $revision = Revision::newFromTitle($title);
>>   $user = User::newFromId($revision->getUser());
>>
>> if(($user->isAnon()==false)&&($wgCommandLineMode==false))://bug es wird
> trotzdem geadded bei SMW_refreshData.php
>>   $dataItem =
> SMWDIWikiPage::newFromTitle($user->getUserPage());
>>   $property = new SMWDIProperty('___SDWED');
>>   if(!is_null($dataItem)):
>>   $data->addPropertyObjectValue($property, $dataItem);
>>   endif;
>>   endif;
>>
>>
>>> The Factbox is populated while parsing the page. Every meta property
>>> that
>> is added at this time will appear in the Factbox. As a matter of
>>> principle, m

Re: [SMW-devel] Any chance to get special properties to the factbox?

[Markus Krötzsch]

On 23/02/13 17:36, Lists wrote:
> Hello Markus and the others,
>
>
> we are doing this in a "SMWStore::updateDataBefore" - handler. The property
> shows up in the browse function not in the "normal" article view. If this
> isn't functional how can we alter an existing property value with this
> handler?
> Let's say we have a property called "Edited by". This property should have
> the value (string) "Hans Bot" which would be set programatically in an
> "SMWStore::updateDataBefore - handler" while saving an article. How to
> archieve this?

Properties that should appear in the Factbox need to be added during 
parsing, so you would have to use another MediaWiki hook that is 
triggered during parsing. The SMWStore hook is called only when the page 
is saved, so it does not happen at all on preview, so it cannot have any 
effect on the Factbox. There are many MW hooks that run during parsing, 
but we do not have a standard pattern or example code for hacking into 
the system at this point, I think. You can have a look at the SMW code 
if you like (SMWParseData implements the main methods used to store data 
during parsing; most importantly, you can get the SMWSemanticData object 
during parsing by calling SMWParseData::getSMWData( $parser ); if you 
store data in there during parsing then it should appear in the Factbox).

Cheers,

Markus

>
> For now we have this for a special/meta property which doesn't fits:
>
>//Edited by
>  $revision = Revision::newFromTitle($title);
>  $user = User::newFromId($revision->getUser());
>  if(($user->isAnon()==false)&&($wgCommandLineMode==false))://bug es
> wird trotzdem geadded bei SMW_refreshData.php
>  $dataItem = SMWDIWikiPage::newFromTitle($user->getUserPage());
>  $property = new SMWDIProperty('___SDWED');
>  if(!is_null($dataItem)):
>  $data->addPropertyObjectValue($property, $dataItem);
>  endif;
>  endif;
>
>
>> The Factbox is populated while parsing the page. Every meta property that
> is added at this time will appear in the Factbox. As a matter of >principle,
> meta properties that are not known yet when building the page cannot be
> shown in the Factbox (simple way to judge this: is the >property already
> known when creating an edit preview?).
>
>> If a property is added in time and still does not appear in the Factbox, it
> might be that its display is disabled. This is defined when >registering new
> properties. One can also check if the property is shown in Special:Browse
> (if not, then it may be invisible; maybe it does >not even have a
> user-readable label).
>
>> Cheers
>
>> Markus
>
> 
> 
> Hello SMW community,
>
>
> I know it is sometimes a little bit hard to get response from this list. But
> I will try it again with a longer description.
> We are working with SMW and MW, contributed extensions to MW since several
> years now and want to develope some special SMW-based MW-extensions. We are
> not able to visit SMW-Conferences- so we only can reach the currently
> published mailinglists, documentations and videos.
>
> Thanks to the extension "Semantic Extra Special Properties" it is now
> transparent how to create properties with PHP for the first time by a simple
> and god to use example code. We also managed to read, delete properties and
> add cumulative values to a property (maybe the wrong way). To get a little
> bit progress in this CRUD-related cases I want to aks the following
> questions in the hope someone can provide anwers/resources. I post no code
> here because my last mail was truncated and trashed ;-).
>
> 1. Unfortunately we did not succeed in using the SMWWriter extension. Is
> this project still maintained. Did someone used it with MW 1.20.x and SMW
> 1.8.x?
>
> 2. When we are dealing with PHP-driven properties, we always produce special
> properties. How can we create and manipulate properties related to userland
> in PHP?
>
> 2.1 How can we avoid doubling of values in a string typed property?
>
> 2.2 How can we influence the display of PHP-generated properties in the
> factbox?
>
> 3. Instead of the SF-API we want to use our already established MW-API
> interface and poor PHP. Are there any additional resources how to manage
> this CRUD-things...create, read, update, delete semantic properties on an
> "official standard way" so that we can maintain our developments along this
> official standards in the long term?
>
> best regards and thank you for your help
>
> Steve
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> ___
> Semediawiki-devel mailing list
> S

[SMW-devel] Text is the new String

[Markus Krötzsch]

Dear developers,

change https://gerrit.wikimedia.org/r/50488 merges the String and Text 
datatypes on all levels (user labels, datatype ids, PHP objects). This 
means the following for your future code:

* Use SMWDIBlob instead of SMWDIString
* Use '_txt' instead of '_str'

For SMW 1.9, '_str' and SMWDIString remain functional as aliases for 
'_txt' and SMWDIBlob, but this will have to end at some point. So code 
should gradually be updated to replace the old identifiers. No interface 
changes, so it is really just a search & replace activity.

Cheers,

Markus

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Any chance to get special properties to the factbox?

[Markus Krötzsch]

On 20/02/13 14:09, Lists wrote:
>
> Hello,
>
> is there any chance to get special properties (meta) into the factbox?

The Factbox is populated while parsing the page. Every meta property 
that is added at this time will appear in the Factbox. As a matter of 
principle, meta properties that are not known yet when building the page 
cannot be shown in the Factbox (simple way to judge this: is the 
property already known when creating an edit preview?).

If a property is added in time and still does not appear in the Factbox, 
it might be that its display is disabled. This is defined when 
registering new properties. One can also check if the property is shown 
in Special:Browse (if not, then it may be invisible; maybe it does not 
even have a user-readable label).

Cheers

Markus


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] SMWSQLStore2 support will be dropped

[Markus Krötzsch]

Dear all,

this email is to confirm that the old database management code 
"SQLStore2" (which was superseded by "SQLStore3" in SMW 1.8) will be 
dropped in future SMW versions. This has been on our public roadmap for 
some time -- I am just sending this to make everyone aware of this to 
avoid surprises.

This only affects you if you run a wiki that uses SMW 1.7 or older, or 
if you use one of the following lines in its LocalSettings:

$smwgDefaultStore = 'SMWSQLStore2';
SMWSparqlStore::$baseStoreClass = 'SMWSQLStore2';

If this is the case for you, you will need to remove the above lines 
when upgrading SMW, run the setup again (like on normal installation), 
and refresh the data as explained in the documentation.

Dropping the old code will allow us to implement a number of 
improvements in SMW that would not work with the old code. Most notably 
the String and Text datatypes will become the same.

Cheers,

Markus

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Print requests in SMWDescription

[Markus Krötzsch]

Hi,

On 05/02/13 15:21, Jeroen De Dauw wrote:
> Hey,
>
> (Question mainly aimed at Markus, though if you have any idea, I'd be
> happy to hear it)
>
> Why do all SMWDescription objects have an array of print requests?
>
> In my mind a query consists out of
>
> * A number of SMWDescription objects specifying the condition
> * A number of print requests specifying what to select
> * A number of parameters, some of which affect the query, most of which
> are for post-query processing
>
> So I am a bit surprised to find that the SMWDescription base class holds
> an array of print requests.

To answer this question, we have to go into the history of SMW ... There 
was a time when the world was young and print requests were still part 
of queries, written using [[Property::*]]. This is why they were located 
"inside" the query structure at this time. Today, all printout requests 
should be what the code calls "extra printouts", which are given as a 
separate list.

At all times, SMW just pushed print requests down to the root of the 
query for evaluation, so the position they were found in did not matter. 
One could have conceived another approach where one can really print 
information about objects that are referred to in the query. For example:

"show all employees that work for charity organisations, together with 
the size of this charity"

would be a print request about the organisation that has "caused" an 
employee to appear in the result. This is different from complex print 
requests that we are discussing now, where you could ask "show all 
employees that work for charity organisations, together with the size of 
all organisations (charity or not) that they work for".

To implement the former semantics, one would have to consider queries to 
return more than just a list of pages (employees), but rather a list of 
tuples (employees, charity organisations). This would lead to new issues 
(combinatorial explosion where the same employee can occur in many 
result rows; mismatch with the expectations of most query result 
printers). We had this at the very beginning of SMW and gave it up 
because it was not what the user wanted in most cases. So there simply 
are no real "nested printouts" now and it would be hard to add this 
functionality to the #ask stack.

Markus


--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] [News] SMW 1.8 released

[Markus Krötzsch]

Dear all,

we are happy to announce the release of the new Semantic MediaWiki 
version SMW 1.8 [1]. A record number of people have contributed to this 
release, and a large number of changes have been made, both visible and 
internal. Major new features of SMW 1.8 include:

* New data storage: more efficient, less database activity
* Interface improvements in Special:Ask
* New atom and RSS printer
* SMW now provides information about #ask queries used on the wiki [2]
* Support for #subobjects without a name
* Ask API now stable, with added continuation support

Moreover, a large number of bugs have been fixed, compatibility to all 
recent MW versions has been ensured, a lot of internal improvements have 
been made, and translations have been added. Many internal changes 
prepare further improvements in upcoming versions. The great new 
Semantic Result Formats 1.8 also is based on SMW 1.8 -- it will be 
released shortly.


== Extensions for SMW 1.8 ==

Latest versions of major SMW extensions are compatible with the new 
release. In particular this involves Semantic Forms, Semantic Maps 2.0, 
Semantic Drilldown 1.2.4, Semantic Internal Objects 0.7, and the amazing 
new Semantic Result Formats 1.8. There are known issues with "distance 
queries" in S Maps and with "remote autocompletion" in S Forms.


== Installation and Upgrade ==

To get the new version, you can download the file package [3] (includes 
the required version 0.5.1 of the Validator extension) or pull directly 
from git using the tag "1.8" [4]. Installation instructions are at [5]. 
Users who upgrade existing installations will need to refresh their data 
(you can continue using the old data until this is done). It is not 
difficult, but please do read the upgrade instructions [5].

Users who are now running the git master branch should be aware that 
this branch will soon become the place of SMW 1.9 development, and may 
thus be less stable than in recent weeks. You can switch to the release 
tag "1.8" for stability [4].


== The most important bit ==

SMW 1.8 was a team effort that had an amazing amount of support from too 
many people to list here. We would not have this release without the 
significant contributions from James "Hong Kong" and Nischay Nahata. 
Karsten Hoffmeyer once more did a great job in getting the documentation 
ready for the release. Further thanks are due to the Wikimedia 
Foundation, Wikimedia Germany e.V., Google Inc., and to the University 
of Oxford who each supported some of us in doing this work.


We hope you enjoy the new release.

Cheers,

Markus & Jeroen


[1] http://semantic-mediawiki.org/wiki/Semantic_MediaWiki_1.8.0
[2] http://semantic-mediawiki.org/wiki/Help:Special_property_Has_query
[3] 
http://sourceforge.net/projects/semediawiki/files/semediawiki/Semantic%20MediaWiki%201.8/
[4] http://semantic-mediawiki.org/wiki/Help:Download#Git
[5] http://semantic-mediawiki.org/wiki/Help:Installation

--
Keep yourself connected to Go Parallel: 
DESIGN Expert tips on starting your parallel project right.
http://goparallel.sourceforge.net/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] MediaWiki compatibility for 1.9

[Markus Krötzsch]

+1 for dropping 1.18 in 1.9.

Markus

On 02/12/12 02:43, James HK wrote:
> Hi,
>
> Thanks. You read my mind, I was just about to write a similar email as
> it happens 1.18 already dropped from the official MW supporting list
> [1].
>
> [1] http://www.mediawiki.org/wiki/Download
>
> Cheers,
>
> On 12/2/12, Jeroen De Dauw  wrote:
>> Hey,
>>
>> Since the 1.8 release got relayed quited a bit (it was originally scheduled
>> for September and will be made tomorrow) I suggest we do not just drop
>> support for MediaWiki 1.17 but for 1.18 as well. Assuming we manage to
>> stick to the release schedule for 1.9, MediaWiki 1.19 will have been
>> available for almost a year at the point of release. Any objections? If
>> not, I'll update the roadmap [0].
>>
>> [0] https://semantic-mediawiki.org/wiki/Roadmap#SMW_1.9
>>
>> Cheers
>>
>> --
>> Jeroen De Dauw
>> http://www.bn2vs.com
>> Don't panic. Don't be evil.
>> --
>>
>
> --
> Keep yourself connected to Go Parallel:
> DESIGN Expert tips on starting your parallel project right.
> http://goparallel.sourceforge.net/
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Keep yourself connected to Go Parallel: 
DESIGN Expert tips on starting your parallel project right.
http://goparallel.sourceforge.net/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [News] SMW 1.8 RC1 available

[Markus Krötzsch]

Dear Luka,

thanks for the reports. The second issue has now been fixed in master 
[1] and will thus appear in SMW 1.8.

For the PostgreSQL initialisation, I would like to have more 
information. It seems odd that this should be needed in general to 
create a working PostgreSQL DB. It seems to me that such 
encoding/decoding issues (hex strings) should be handled by the database 
access layer (or by the database), so that it works with any 
configuration. Changing the configuration instead seems like a 
workaround. Maybe this should actually be a bug against the MediaWiki 
PostgreSQL access layer?

Very nice site you have indeed. Would be great to have this as a Wiki of 
the Month at some point. See [2].

Cheers,

Markus

[1] https://gerrit.wikimedia.org/r/36163
[2] http://semantic-mediawiki.org/wiki/Wiki_of_the_Month


On 28/11/12 01:23, Luka Frelih wrote:
> Hello!
>
> We stumbled on two issues with 1.8rc. Both were puzzling and scary at
> first, and each took its own debugging session to tame, now both
> resolved. ;)
>
> First we saw hex-encoded outputs of all textual objects (strings,blobs)
> larger than 72 characters.
> Our site runs postgresql - it turned out to be an incompatible default
> configuration setting on the role (user). The fix is simple and only
> needs to be run once against the database:
>
> ALTER ROLE {your_db_username_here} SET bytea_output = 'escape';
>
> The other and it seems default option is 'hex' and SMW is not able to
> recognize or deal with it properly. So this configurations seems to be
> necessary. It would be great if this can somehow get set automatically
> upon creating the datastore, when running maintenance/SMWsetup.php.
>
> The other issue manifests when queries return wikipagedataitems
> containing + in their title. This triggers the "Somebody tried to use
> spaces in a cache title!" exception, crashing the page. Fixing it is
> simple in the Exporter function findDataItemForExpElement: use
> rawurldecode instead of 'plain' urldecode, which converts the + to space.
>
> diff --git a/includes/export/SMW_Exporter.php
> b/includes/export/SMW_Exporter.php
> index 9da3f1a..b56f5b3 100644
> --- a/includes/export/SMW_Exporter.php
> +++ b/includes/export/SMW_Exporter.php
> @@ -326,7 +326,7 @@ class SMWExporter {
> $wikiNamespace = self::getNamespaceUri( 'wiki' );
> if ( strpos( $uri, $wikiNamespace ) === 0 ) {
> $localName = substr( $uri, strlen( $wikiNamespace ) );
> - $dbKey = urldecode( self::decodeURI( $localName ) );
> + $dbKey = rawurldecode( self::decodeURI( $localName ) );
>
> $parts = explode( '-23', $dbKey, 2 );
> if ( count( $parts ) == 2 ) {
>
> Now that bugs are fixed, am looking forward to finally try out
> subobjects and what else the new SMW has to offer! :)
>
> The site: http://www.culture.si/ - powered by SMW, perhaps not a bad
> usecase.
>
> best,
> LF
>
>
> Dne 22. 11. 2012 08:41, piše Markus Krötzsch:
>> Dear all,
>>
>> SMW 1.8 is close to its first stable release. We have now packaged a
>> Release Candidate for download [1]. If all goes well, this will become
>> SMW 1.8.0 in about a week. As usual, the package includes the required
>> version of Validator.
>>
>> Major new features include:
>>
>> * New data storage: more efficient, less database activity
>> * Interface improvements in Special:Ask
>> * New atom and RSS printer
>> * SMW now provides information about #ask queries used on the wiki [2]
>> * Support for #subobjects without a name
>> * Ask API now stable, with added continuation support
>>
>> Moreover, a large number of bugs have been fixed, compatibility to all
>> recent MW versions has been ensured, a lot of internal improvements have
>> been made, and translations have been added. A record number of people
>> have contributed to this work.
>>
>> For those of you who download versions from git, there is also an
>> according tag there for this RC. The master branch itself will remain in
>> a stable state until 1.8 is released.
>>
>> Feedback is most welcome.
>>
>> Cheers,
>>
>> Markus
>>
>>
>> [1]
>> http://sourceforge.net/projects/semediawiki/files/semediawiki/Semantic%20MediaWiki%201.8%20RC1/
>> [2] You can now query for queries. Examples:
>>
>> * The 10 largest queries used on the wiki with their size and query string:
>>
>> {{#ask:[[query string::+]]
>> |?query size
>> |?query string
>> |sort=query size
>> |order=DESC
>> |limit=10
>> }}
>>
>> * The number of queries that use the format "list":
>>
>> {{#ask:[[query format::list]] |f

Re: [SMW-devel] [Semediawiki-user] s13n stands for semantification

[Markus Krötzsch]

On 18/11/12 13:30, Yury Katkov wrote:
> Ok, I understand that: the choice is between to store or not to store
> the source of the data by grouping the triples in graphs. Of course
> it's up to core developers to choose.
> Another set of questions:
> 1) how the s13n's programming interfaces for extension developers will
> look like?

Specifying this is a major part of the task.

> 2) what is the priority of this task?

I would say between 17 and 18.

My (serious) point here is: priority must be relative. We have many 
requests for 1.9 (e.g. printouts for property chains and aggregates, 
paged query results, removing length restrictions on URL values, ...). 
The question is: what is the priority for doing this *before* any of 
these other, more long-standing requests? I think this can only be 
decided by collecting the main todos and then placing them in an order.

> 3) Is it possible to sponsor the development to make it quicker? I
> think that the companies interested in this feature can build together
> a fund/grant to support the volunteer. WikiVote can definitely take
> part in that sponsorship campaign.

Thanks. A sponsored developer could help with specifying the 
requirements and producing technical documentation. The actual 
implementation is only possible after we know what we want to do. The 
project should start by collecting use-cases so that we get an idea of 
what we would need and how this new functionality could be invoked.

Technically, the changes are blocked only by other changes in the store 
that are more urgent (further simplification of the table layout and 
data model, completion of the diff code). No other dependencies with 
ongoing SMW work is obvious to me now.

Markus

> -
> Yury Katkov, WikiVote
>
>
>
> On Fri, Nov 16, 2012 at 9:35 PM, Jeroen De Dauw  
> wrote:
>> Hey,
>>
>>
>>> Can you clarify the idea of named graphs a little bit? And what
>>> storage overhead we have in them?
>>
>>
>> Instead of having rows such as
>>
>> ( subject id, property id, value )
>>
>> in the store, you instead have
>>
>> ( subject id, property id, value, graph id )
>>
>> Of course our schema is more complex then this, but it's the same idea.
>>
>> Cheers
>>
>> --
>> Jeroen De Dauw
>> http://www.bn2vs.com
>> Don't panic. Don't be evil.
>> --
>
> --
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query formats min and max

[Markus Krötzsch]

On 23/11/12 10:15, Jeroen De Dauw wrote:
> Hey,
>
>> Interesting. How does that work?
>
> Use the source, Markus... :)
>
> https://github.com/wikimedia/mediawiki-extensions-SemanticResultFormats/blob/master/formats/math/SRF_Math.php#L83

Not very helpful. I know I can dig into source code to find out how 
everything works. If somebody already knows what this feature does, I am 
still happy to read a (short) synopsis. Would also make sense to 
document this somewhere.

Markus

>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil.
> --
>
> --
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query formats min and max

[Markus Krötzsch]

On 22/11/12 11:22, Yury Katkov wrote:
> The premises are clear:
> (1) the current implementation of the max parser function is slow and
> (2)  there is a workaround for making max queries quicker.
>
> The conclusion is not clear: "let's drop the max ASAP".
>
> It's not that hard to replace the current implementation of MAX format
> with the faster one and save the backward compatibility there.

It would be hard to do this (cleanly) in an automated way, because it 
requires changes to many other query parameters that are superior to 
format (format is the last thing to play a role in query evaluation). 
Also, as Jeroen illustrated, there are cases where the user really wants 
to have a "local" maximum, and we would not be able to recognise these. 
So let us just keep it as it is but improve the docs (maybe this is what 
you meant).

Markus


> -
> Yury Katkov, WikiVote
>
>
>
> On Thu, Nov 22, 2012 at 1:23 PM, Markus Krötzsch
>  wrote:
>> Hi,
>>
>> I would like to ask about this:
>>
>> http://semantic-mediawiki.org/wiki/Help:Max_format
>>
>> I am afraid to say that this idea seems to be fundamentally broken. The
>> above page seriously suggests to find the largest population number in
>> the wiki by querying for a list of *all cities with and without
>> population* and invoke PHP code that scans through this list to find the
>> maximum (this is what format=max does, AFAIK). The query to do this is:
>>
>> {{#ask: [[Category:City]]
>> | ?Population
>> | format=max
>> }}
>>
>> This is an extremely slow method of producing wrong results (the results
>> will be wrong as soon as there are enough pages in the wiki so that the
>> one with the maximum value is after the default query limit when
>> ordering results alphabetically).
>>
>> What one would do instead is to ask for the one result that has the
>> largest value right away, like this:
>>
>> {{#ask: [[Category:City]]
>> | ?Population
>> | sort=population
>> | order=DESC
>> | limit=1
>> | format=max
>> }}
>>
>> The max format in this case is obsolete, since one could also just do
>>
>> {{#ask: [[Category:City]]
>> | ?Population=
>> | mainlabel=-
>> | sort=population
>> | order=DESC
>> | limit=1
>> }}
>>
>> This has the big advantage that one can also use further output
>> formatting on the resulting number, e.g., to get it in a plain format
>> without any beautification.
>>
>>
>> I just noted these problems since there seem to be cases where PHP runs
>> out of time/memory due to users following the above query anti-pattern
>> [1]. My conclusion would be: let's drop max/min as soon as possible and
>> change the documentation to give the efficient query pattern I gave above.
>>
>> Markus
>>
>> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=42347
>>
>> --
>> Monitor your physical, virtual and cloud infrastructure from a single
>> web console. Get in-depth insight into apps, servers, databases, vmware,
>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>> Pricing starts from $795 for 25 servers or applications!
>> http://p.sf.net/sfu/zoho_dev2dev_nov
>> ___
>> Semediawiki-devel mailing list
>> Semediawiki-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query formats min and max

[Markus Krötzsch]

On 22/11/12 14:44, Jeroen De Dauw wrote:
> Hey,
>
> Markus, I agree with you that in most cases you're better off doing a
> sorted query, and think this should be reflected in the documentation.
> I however would not be to quick to conclude the max format is
> completely useless. For one thing it supports recursion when dealing
> with containers.

Interesting. How does that work?

> People might also be relying on the result being the
> max or min of the result set and not everything that matches the
> query, ie querying the 10 last edited cities and getting the biggest
> population of those.

Nice example. This is also a good example to illustrate how max really 
works (finding the max within the displayed results as opposed to 
finding the max in the database).

>
> So I suggest making sure the documentation is clear enough so people
> don't misuse it while keeping the printer. After all, it's in SRF and
> can easily be disabled. And it has not been causing serious problems
> motivating removal to my knowledge.

Ok, agreed. We can achieve all of this with good documentation. Maybe it 
is even better to have max as an anchor for this documentation -- people 
who don't want max (but don't know it yet) will likely be looking there.

Markus



--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [News] SMW 1.8 RC1 available

[Markus Krötzsch]

On 22/11/12 10:19, Neill Mitchell wrote:
> Hi all.
>
> One thing to note. You must run the SMW_refreshData maintenance script
> after upgrading. I had a brief moment of panic as all my property values
> were empty!

Yes, please *do* read the section about upgrading in INSTALL. I paste 
the main part of its text below, just in case you cannot find INSTALL ;-)

Also, please do not try to delete your old SQLStore2 yet (a bug has been 
discovered there today; the script won't accept the parameter that 
specifies the store, so it only let's you delete the store you currently 
use). This will be re-enabled before the release.

Markus

===

SMW 1.8 introduces a new default database layout (SMWSQLStore3). You can
continue to use the old layout (SMWSQLStore2) for transition purposes, 
but it
is strongly recommended to migrate to the new layout for future 
compatibility.

To use the old layout at first, add $smwgDefaultStore = 'SMWSQLStore2'; to
your LocalSettings.php. If you are using $smwgDefaultStore = 
'SMWSparqlStore';
already, then you should keep this line and add the following:
SMWSparqlStore::$baseStoreClass = 'SMWSQLStore2'; (after enableSemantics).
After setting these, run SMW_setup.php or Special:SMWAdmin upgrade as usual.
The wiki should now work as normal, but using the old storage structures.

To migrate to the new store, you can do a normal "full refresh" for the new
store. Run the following two commands (both will run a while).

php SMW_refreshData.php -v -b SMWSQLStore3 -fp
php SMW_refreshData.php -v -b SMWSQLStore3

The running wiki will not be affected by this, but the operation could 
affect
server speed. See
http://semantic-mediawiki.org/wiki/Help:Repairing_SMW%27s_data for details.

After successful migration, remove the lines with 'SMWSQLStore2' from your
LocalSettings.php to use the new store. You can always return to the old 
store
in case of problems. If the old store is no longer needed, it can be deleted
(and its memory freed) by running

...
(this does not work, as mentioned above; don't try it yet)



>
> Cheers
> Neill.
>
> On 22/11/12 10:00, Neill Mitchell wrote:
>> Hi Markus.
>>
>> This is great news. Thanks to all involved!
>>
>> I'll run it through my regression test suite over the next couple of days.
>>
>> Best regards
>> Neill.
>> On 22/11/12 07:41, Markus Krötzsch wrote:
>>> Dear all,
>>>
>>> SMW 1.8 is close to its first stable release. We have now packaged a
>>> Release Candidate for download [1]. If all goes well, this will become
>>> SMW 1.8.0 in about a week. As usual, the package includes the required
>>> version of Validator.
>>>
>>> Major new features include:
>>>
>>> * New data storage: more efficient, less database activity
>>> * Interface improvements in Special:Ask
>>> * New atom and RSS printer
>>> * SMW now provides information about #ask queries used on the wiki [2]
>>> * Support for #subobjects without a name
>>> * Ask API now stable, with added continuation support
>>>
>>> Moreover, a large number of bugs have been fixed, compatibility to all
>>> recent MW versions has been ensured, a lot of internal improvements have
>>> been made, and translations have been added. A record number of people
>>> have contributed to this work.
>>>
>>> For those of you who download versions from git, there is also an
>>> according tag there for this RC. The master branch itself will remain in
>>> a stable state until 1.8 is released.
>>>
>>> Feedback is most welcome.
>>>
>>> Cheers,
>>>
>>> Markus
>>>
>>>
>>> [1]
>>> http://sourceforge.net/projects/semediawiki/files/semediawiki/Semantic%20MediaWiki%201.8%20RC1/
>>> [2] You can now query for queries. Examples:
>>>
>>> * The 10 largest queries used on the wiki with their size and query string:
>>>
>>> {{#ask:[[query string::+]]
>>> |?query size
>>> |?query string
>>> |sort=query size
>>> |order=DESC
>>> |limit=10
>>> }}
>>>
>>> * The number of queries that use the format "list":
>>>
>>> {{#ask:[[query format::list]] |format=count }}
>>>
>>> * The number of pages that use queries that use the format "list":
>>>
>>> {{#ask:[[has query.query format::list]] |format=count }}
>>>
>>> * The total number of queries:
>>>
>>> {{#ask:[[query size::+]] |format=count }}
>>>
>>> * Pages that have a query with format list and one with 

[SMW-devel] Query formats min and max

[Markus Krötzsch]

Hi,

I would like to ask about this:

http://semantic-mediawiki.org/wiki/Help:Max_format

I am afraid to say that this idea seems to be fundamentally broken. The 
above page seriously suggests to find the largest population number in 
the wiki by querying for a list of *all cities with and without 
population* and invoke PHP code that scans through this list to find the 
maximum (this is what format=max does, AFAIK). The query to do this is:

{{#ask: [[Category:City]]
| ?Population
| format=max
}}

This is an extremely slow method of producing wrong results (the results 
will be wrong as soon as there are enough pages in the wiki so that the 
one with the maximum value is after the default query limit when 
ordering results alphabetically).

What one would do instead is to ask for the one result that has the 
largest value right away, like this:

{{#ask: [[Category:City]]
| ?Population
| sort=population
| order=DESC
| limit=1
| format=max
}}

The max format in this case is obsolete, since one could also just do

{{#ask: [[Category:City]]
| ?Population=
| mainlabel=-
| sort=population
| order=DESC
| limit=1
}}

This has the big advantage that one can also use further output 
formatting on the resulting number, e.g., to get it in a plain format 
without any beautification.


I just noted these problems since there seem to be cases where PHP runs 
out of time/memory due to users following the above query anti-pattern 
[1]. My conclusion would be: let's drop max/min as soon as possible and 
change the documentation to give the efficient query pattern I gave above.

Markus

[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=42347

--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] [News] SMW 1.8 RC1 available

[Markus Krötzsch]

Dear all,

SMW 1.8 is close to its first stable release. We have now packaged a 
Release Candidate for download [1]. If all goes well, this will become 
SMW 1.8.0 in about a week. As usual, the package includes the required 
version of Validator.

Major new features include:

* New data storage: more efficient, less database activity
* Interface improvements in Special:Ask
* New atom and RSS printer
* SMW now provides information about #ask queries used on the wiki [2]
* Support for #subobjects without a name
* Ask API now stable, with added continuation support

Moreover, a large number of bugs have been fixed, compatibility to all 
recent MW versions has been ensured, a lot of internal improvements have 
been made, and translations have been added. A record number of people 
have contributed to this work.

For those of you who download versions from git, there is also an 
according tag there for this RC. The master branch itself will remain in 
a stable state until 1.8 is released.

Feedback is most welcome.

Cheers,

Markus


[1] 
http://sourceforge.net/projects/semediawiki/files/semediawiki/Semantic%20MediaWiki%201.8%20RC1/
[2] You can now query for queries. Examples:

* The 10 largest queries used on the wiki with their size and query string:

{{#ask:[[query string::+]]
|?query size
|?query string
|sort=query size
|order=DESC
|limit=10
}}

* The number of queries that use the format "list":

{{#ask:[[query format::list]] |format=count }}

* The number of pages that use queries that use the format "list":

{{#ask:[[has query.query format::list]] |format=count }}

* The total number of queries:

{{#ask:[[query size::+]] |format=count }}

* Pages that have a query with format list and one with format table:

{{#ask: [[has query.query format::list]] [[has query.query 
format::table]] }}

--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] s13n stands for semantification

[Markus Krötzsch]

On 16/11/12 14:32, Yury Katkov wrote:
> Hi Markus!
>
> That's very nice! How hard/long is that task?

Unknown. First we need to decide what we want to have.

>
> We can now use something like SMWStore::updateDataBefore hook, but not
> good approach sometimes: with voting and history s13n you'll need to
> create a lot of property values each time the data is updated.  When
> you semantify the page history you'll need to create several hundreds
> datavalues representing the revisions of the current page - that's too
> exprensive.

The idea was to have data that is not governed by page updates. There 
are two basic designs:

(1) We introduce something like "named graphs". All data belongs to a 
graph. Only data in the default graph is updated with the page contents; 
other writing methods are provided for the rest. Maximally flexible, but 
some storage overhead.

(2) We do not use graphs but bind this information to properties: some 
properties are governed by page contents, other properties are governed 
in other ways. Less flexible, but no storage overhead.

Cheers,

Markus

>
> -
> Yury Katkov, WikiVote
>
>
>
> On Fri, Nov 16, 2012 at 4:34 PM, Markus Krötzsch
>  wrote:
>> On 16/11/12 11:27, Yury Katkov wrote:
>>>
>>> Hi everyone!
>>>
>>> I talked about the topic of semanticfication of the MediaWiki
>>> extension with a lot of people in SMWCon and saw the great interest to
>>> this topic.
>>>
>>> We have tried to describe the problem on this page:
>>>
>>> http://semantic-mediawiki.org/wiki/User:Yury_Katkov/s13n_stands_for_semantification
>>>
>>> To the users: if you're interested in topics of s13n and have the use
>>> case that is not described there - please describe it.
>>>
>>> To the developers: let's think together what's needed to be done in
>>> the SMW core to make s13n of the extensions cleaner and easier. Also
>>> the rough time/effort  estimate would be great (maybe SMW companies
>>> can sponsor the development?)
>>
>>
>> I have discussed this already with Jeroen. We can help to provide an
>> infrastructure for this. However, the access paths for enabling efficient
>> #ask queries are different from the traditional MW access paths to metadata,
>> so all data that is to be s8ed (;-) will need to be replicated in SMW
>> tables, even if the same MySQL DB is used.
>>
>> Cheers,
>>
>> Markus
>>
>>>
>>> Cheers,
>>> Yury Katkov, WikiVote
>>>
>>>
>>> --
>>> Monitor your physical, virtual and cloud infrastructure from a single
>>> web console. Get in-depth insight into apps, servers, databases, vmware,
>>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>>> Pricing starts from $795 for 25 servers or applications!
>>> http://p.sf.net/sfu/zoho_dev2dev_nov
>>> ___
>>> Semediawiki-devel mailing list
>>> Semediawiki-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>>
>>
>
> --
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] s13n stands for semantification

[Markus Krötzsch]

On 16/11/12 11:27, Yury Katkov wrote:
> Hi everyone!
>
> I talked about the topic of semanticfication of the MediaWiki
> extension with a lot of people in SMWCon and saw the great interest to
> this topic.
>
> We have tried to describe the problem on this page:
> http://semantic-mediawiki.org/wiki/User:Yury_Katkov/s13n_stands_for_semantification
>
> To the users: if you're interested in topics of s13n and have the use
> case that is not described there - please describe it.
>
> To the developers: let's think together what's needed to be done in
> the SMW core to make s13n of the extensions cleaner and easier. Also
> the rough time/effort  estimate would be great (maybe SMW companies
> can sponsor the development?)

I have discussed this already with Jeroen. We can help to provide an 
infrastructure for this. However, the access paths for enabling 
efficient #ask queries are different from the traditional MW access 
paths to metadata, so all data that is to be s8ed (;-) will need to be 
replicated in SMW tables, even if the same MySQL DB is used.

Cheers,

Markus

>
> Cheers,
> Yury Katkov, WikiVote
>
> --
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Preparing the SMW 1.8 release

[Markus Krötzsch]

Additional note on RDF store support in SMW 1.8:

Since a moment ago, the SMWSparqlStore has been changed to use the new 
SMWSQLStore3 as its SQL backend (the old used SQLSTore2). You can change 
it back to use the old store instead by setting:

SMWSparqlStore::$baseStoreClass = 'SMWSQLStore2';

in your LocalSettings (after enableSemantics(), or it will complain that 
the class SMWSparqlStore is not known).

Cheers,

Markus

On 11/11/12 17:01, Markus Krötzsch wrote:
> Dear developers,
>
> after two weeks of intense bugfixing, we are now getting close to the
> actual SMW 1.8 release. We will probably have another release candidate
> packaged soon, but as developers you might want to try the git version
> now, especially to check if it runs with your own code and data.
>
>
> == Installation ==
>
> Installation is as usual, as if installing SMW from scratch
> (Special:SMWAdming: setup database, refresh data). Before doing this,
> add the following to your LocalSettings.php:
>
> $smwgDefaultStore = 'SMWSQLStore3';
>
> This will be the default when SMW 1.8 is released, but not yet. You can
> also setup/refresh SQLStore3 before enabling it in LocalSettings -- see
> below.
>
>
> == Upgrade from earlier SMW 1.8 versions ==
>
> If you have tested development versions or RCs of SMW 1.8 before, and
> you are already using "$smwgDefaultStore = 'SMWSQLStore3';", please
> delete the old Store3 completely first. You can do this by running the
> maintenance script "php SMW_setup.php --delete". After this, you can do
> the normal Special:SMWAdmin setup and refresh as above.
>
> If you do not delete Store3 first, it should still work without errors,
> but your database will retain dead contents that will never be deleted.
>
>
> == Set up new store while still using SQLStore2 ==
>
> To fully initialise SMWSQLStore3 *before* activating it in
> LocalSettings.php, you can do:
>
> * first run "php SMW_refreshData.php -b SMWSQLStore3 -fp" to recreate
> tables and initialise property pages,
> * then run "php SMW_refreshData.php -b SMWSQLStore3" again to refresh
> all normal pages.
>
> This can be done before switching to SQLStore3 (the "-b SMWSQLStore3"
> tells SMW to use this store, whatever you use right now). The "-f" in
> the first call makes sure that any existing Store3 data is deleted, so
> you can also use this if you used SQLStore3 before.
>
>
> Cheers,
>
> Markus
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_nov
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] Preparing the SMW 1.8 release

[Markus Krötzsch]

Dear developers,

after two weeks of intense bugfixing, we are now getting close to the 
actual SMW 1.8 release. We will probably have another release candidate 
packaged soon, but as developers you might want to try the git version 
now, especially to check if it runs with your own code and data.


== Installation ==

Installation is as usual, as if installing SMW from scratch 
(Special:SMWAdming: setup database, refresh data). Before doing this, 
add the following to your LocalSettings.php:

$smwgDefaultStore = 'SMWSQLStore3';

This will be the default when SMW 1.8 is released, but not yet. You can 
also setup/refresh SQLStore3 before enabling it in LocalSettings -- see 
below.


== Upgrade from earlier SMW 1.8 versions ==

If you have tested development versions or RCs of SMW 1.8 before, and 
you are already using "$smwgDefaultStore = 'SMWSQLStore3';", please 
delete the old Store3 completely first. You can do this by running the 
maintenance script "php SMW_setup.php --delete". After this, you can do 
the normal Special:SMWAdmin setup and refresh as above.

If you do not delete Store3 first, it should still work without errors, 
but your database will retain dead contents that will never be deleted.


== Set up new store while still using SQLStore2 ==

To fully initialise SMWSQLStore3 *before* activating it in 
LocalSettings.php, you can do:

* first run "php SMW_refreshData.php -b SMWSQLStore3 -fp" to recreate 
tables and initialise property pages,
* then run "php SMW_refreshData.php -b SMWSQLStore3" again to refresh 
all normal pages.

This can be done before switching to SQLStore3 (the "-b SMWSQLStore3" 
tells SMW to use this store, whatever you use right now). The "-f" in 
the first call makes sure that any existing Store3 data is deleted, so 
you can also use this if you used SQLStore3 before.


Cheers,

Markus


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SIO - Having trouble querying Semantic Internal Objects

[Markus Krötzsch]

On 10/11/12 19:05, Marcelo Chiaradía wrote:
> Hi Yaron,
>
> I finally discover what was wrong. It seems that if you defined your the
> property in the "#set_internal:", and you start it with a lower case,
> and you create the property after that, for some reason the property
> created start with an upper case, and the semantic search doesn't
> retrieve any result.
>
> To make it work I just replace "#set_internal:hasAnInternalObject" by
> "#set_internal:HasAnInternalObject", and it works perfectly now.

All properties in SMW internally start upper case, and it does not 
matter whether you use upper or lower case on input (SMW's input 
processing will turn it into upper case internally). The fact that this 
does not happen here seems to be a bug in SIO.

Markus

>
> Now I have another question about queries:
>
> is there some way to query some information about my page, based in its
> internal object attributes?
>
> I mean, cause the relationship goes from the internal object to the
> page, and not in the inverse order.
>
> I would like to query all the pages that have an internal object, which
> has an attribute "att1", for example.
>
> Thanks in advacne,
>
> Marcelo.
>
> 2012/11/9 Marcelo Chiaradía  >
>
> Hi Yaron, thanks a lot for your answer.
>
> The full query is
>
> {{#ask: [[hasAnInternalObject::+]]
> |?language
> }}
>
> "hasAnInternalObject" is defined as property of type Page.
> I have check my databse, and indeed my internal object exists, and
> it has a relationship with my page.( "smw_rels2" table)
>
> The weird thing is that when querying for the "language" property, I
> get the internal object as result, so it must exist. I use this
> query for that:
>
> {{#ask: [[language::English]]
> |?language
> }}
>
> Thanks a lot in advance,
>
> Marcelo.
>
>
> 2012/11/9 Yaron Koren mailto:ya...@wikiworks.com>>
>
> Hi Marcelo,
>
> Yes, the first property goes from the internal object to the
> page - which means that "hasAnInternalObject" is not an ideal
> name for it. but nonetheless that query should work. Two questions:
>
> - What is the full query you're calling? Maybe there's an error
> in the syntax.
>
> - Is "hasAnInternalObject" defined as a property of type Page?
>
> -Yaron
>
>
> On Thu, Nov 8, 2012 at 9:22 PM, Marcelo Chiaradía
> mailto:chiaradiamarc...@gmail.com>>
> wrote:
>
> Hello everyone,
>
> Im using the semantic media wiki, with the Semantic Internal
> Objects extension, version 0.6.8.
>
> In one of my pages I have a template, that receives several
> parameters, and the create an internal object with a
> "hasAnInternalObject" property. This is my template:
>
> 
> This is the "MyTemplate" template.
> It should be called in the following format:
> 
> {{MyTemplate
> |Language=
> |Keywords=
> }}
> 
> Edit the page to see the template text.
> 
> {{#set_internal:hasAnInternalObject
> |language={{{Language|}}}
> |keyword#list={{{Keywords}}}
> }}
> 
>
> I can see in my Database that it creates a
> "hasInternalObject" relationship from the internal object to
> the page that defines it. (I supposed that this was gonna be
> from the page to the internal object, but it wasn`t).
>
> Now I would like to query all my internal objects defined
> like before, but I get no results. The query i used for this is
>
> [[hasAnInternalObject::+]]
>
> Any help?
>
> Thanks in advance,
>
> Marcelo.
>
> 
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_nov
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
>
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_nov
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.

Re: [SMW-devel] Help with ask queries - using intermediate results

[Markus Krötzsch]

On 09/11/12 13:34, Marcelo Chiaradía wrote:
> Thanks a lot for your answer. By the way, is there any extension needed
> to run sub-queries?

No, subqueries are a standard part of SMW's query language and all 
storage backends support them. There are also some forms of subqueries 
for which you don't need s:

[[property1.property2::Value]]

is short for

[[property1::[[property2::Value

But this won't work if you have more conditions to check in the subquery 
(e.g.., categories in your case).

Markus


> 2012/11/9 Markus Krötzsch  <mailto:mar...@semantic-mediawiki.org>>
>
> On 09/11/12 09:37, Yury Katkov wrote:
>
> Well you can also use template format if you don't like
> subqueries... It
> would also help you to use the intermediate results.
>
>
> That would let you "see" the intermediate results (otherwise, there
> is no way to find out which of, say, the "has C" values was
> "responsible" for having a line in the result). If you really need
> this, and the information cannot be retrieved later (e.g. if you
> know that there is only one "has C" on any page anyway), then
> templates might help.
>
> However, using templates with queries to format other queries is
> generally a bad approach that is extremely inefficient and will use
> a lot of memory and CPU to display a page (results are fetched from
> the database, rendered as wikitext, re-parsed as queries, which are
> used to fetch more results, ...); it will only work at all of your
> inner results are very limited in number.
>
> Markus
>
>
>
>
> On Wed, Nov 7, 2012 at 5:44 PM, Marcelo Chiaradía
> mailto:chiaradiamarc...@gmail.com>
> <mailto:chiaradiamarcelo@__gmail.com
> <mailto:chiaradiamarc...@gmail.com>>> wrote:
>
>  Hi everyone,
>
>  I'm trying to make an #ask query, using intermediates
> results to do
>  more queries within.
>
>  I have this scenario:
>
>  I have several pages of a category A. Some of them are
> related to
>  other pages of category B, through a "has B" property.
> Then, my B
>  pages are related to pages of category C, through "has C"
> property,
>  and finally my C pages have an attribute "att1", through
> "has att1"
>  property, wich has defined a specific set of values.
>
>  I would like to make an "ask" query, asking for all my A pages,
>  whose must have at least one B page, which must have at
> least one C
>  page, and which must have the "att1 = SOME_VALUE", where
> some value
>  should be a parameter.
>
>  I read about subqueries, doing something like:
>
>  {{#ask: [[Category::A]] [[has B::[[Category::B]] [[has C::
>
>  [[Category::C]] [[has
> att1::{{SOME_VALUE}}
>]]
>  |?att1
>  }}
>
>  Is this the right way to do this? is there a more elegant way?
>
>  One other thing, I would like that if the user doesn't
> complete the
>  "SOME_VALUE" parameter, then return all the pages of
> category A. I
>  tried this with the "+" and "*" characters, but didnt work.
> How is
>  the best way to achieve this?
>
>  Thanks a lot in advance,
>
>  Marcelo.
>
>
> 
> --__--__--
>  LogMeIn Central: Instant, anywhere, Remote PC access and
> management.
>  Stay in control, update software, and manage PCs from one
> command center
>  Diagnose problems and improve visibility into emerging IT
> issues
>  Automate, monitor and manage. Do more in less time with Central
> http://p.sf.net/sfu/__logmein12331_d2d
> <http://p.sf.net/sfu/logmein12331_d2d>
>  _
>  Semediawiki-devel mailing list
> Semediawiki-devel@lists.__sourceforge.net
> <mailto:Semediawiki-devel@lists.sourceforge.net>
>  <mailto:Semediawiki-devel@__lists.sourceforge.net
> <mailto:Semediawiki-devel@lists.sourceforg

Re: [SMW-devel] Help with ask queries - using intermediate results

[Markus Krötzsch]

On 09/11/12 09:37, Yury Katkov wrote:
> Well you can also use template format if you don't like subqueries... It
> would also help you to use the intermediate results.

That would let you "see" the intermediate results (otherwise, there is 
no way to find out which of, say, the "has C" values was "responsible" 
for having a line in the result). If you really need this, and the 
information cannot be retrieved later (e.g. if you know that there is 
only one "has C" on any page anyway), then templates might help.

However, using templates with queries to format other queries is 
generally a bad approach that is extremely inefficient and will use a 
lot of memory and CPU to display a page (results are fetched from the 
database, rendered as wikitext, re-parsed as queries, which are used to 
fetch more results, ...); it will only work at all of your inner results 
are very limited in number.

Markus

>
>
>
> On Wed, Nov 7, 2012 at 5:44 PM, Marcelo Chiaradía
> mailto:chiaradiamarc...@gmail.com>> wrote:
>
> Hi everyone,
>
> I'm trying to make an #ask query, using intermediates results to do
> more queries within.
>
> I have this scenario:
>
> I have several pages of a category A. Some of them are related to
> other pages of category B, through a "has B" property. Then, my B
> pages are related to pages of category C, through "has C" property,
> and finally my C pages have an attribute "att1", through "has att1"
> property, wich has defined a specific set of values.
>
> I would like to make an "ask" query, asking for all my A pages,
> whose must have at least one B page, which must have at least one C
> page, and which must have the "att1 = SOME_VALUE", where some value
> should be a parameter.
>
> I read about subqueries, doing something like:
>
> {{#ask: [[Category::A]] [[has B::[[Category::B]] [[has C::
>   
>   [[Category::C]] [[has att1::{{SOME_VALUE}}
>   ]]
> |?att1
> }}
>
> Is this the right way to do this? is there a more elegant way?
>
> One other thing, I would like that if the user doesn't complete the
> "SOME_VALUE" parameter, then return all the pages of category A. I
> tried this with the "+" and "*" characters, but didnt work. How is
> the best way to achieve this?
>
> Thanks a lot in advance,
>
> Marcelo.
>
> 
> --
> LogMeIn Central: Instant, anywhere, Remote PC access and management.
> Stay in control, update software, and manage PCs from one command center
> Diagnose problems and improve visibility into emerging IT issues
> Automate, monitor and manage. Do more in less time with Central
> http://p.sf.net/sfu/logmein12331_d2d
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_nov
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Help with ask queries - using intermediate results

[Markus Krötzsch]

On 07/11/12 13:44, Marcelo Chiaradía wrote:
> Hi everyone,
>
> I'm trying to make an #ask query, using intermediates results to do more
> queries within.
>
> I have this scenario:
>
> I have several pages of a category A. Some of them are related to other
> pages of category B, through a "has B" property. Then, my B pages are
> related to pages of category C, through "has C" property, and finally my
> C pages have an attribute "att1", through "has att1" property, wich has
> defined a specific set of values.
>
> I would like to make an "ask" query, asking for all my A pages, whose
> must have at least one B page, which must have at least one C page, and
> which must have the "att1 = SOME_VALUE", where some value should be a
> parameter.
>
> I read about subqueries, doing something like:
>
> {{#ask: [[Category::A]] [[has B::[[Category::B]] [[has C::
>   
>   [[Category::C]] [[has att1::{{SOME_VALUE}}
>   ]]
> |?att1
> }}
>
> Is this the right way to do this? is there a more elegant way?

This is the right way. We are open to proposals for nicer looking 
syntax, but the general approach is correct.

>
> One other thing, I would like that if the user doesn't complete the
> "SOME_VALUE" parameter, then return all the pages of category A. I tried
> this with the "+" and "*" characters, but didnt work. How is the best
> way to achieve this?

You need to omit the [[has att1::...]] altogether if you do not want any 
restriction on this property. You can use an #ifeq to do this whenever 
{{SOME_VALUE}} is empty.

Cheers,

Markus



--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_nov
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Exhibit

[Markus Krötzsch]

On 27/10/12 22:21, Jeroen De Dauw wrote:
> Hey,
>
> As most of you probably know, the Exhibit result printer is currently
> rather broken and the code does not really lend itself to be maintained
> or fixed. I suggest just removing it altogether from SRF.

+1
If it does not work, there is no point in keeping it. The upcoming 
filtered printer (for those of you who do not know it yet, see, e.g., 
[1] and [2]) was developed to provide comparable functionality.

Markus

[1] http://www.aroundisleofwight.info/en/What%27s_On
[2] http://www.hampshiregateway.info/en/All_Events

>
> Exhibit 3.0 was released recently [0], so it might be worthwhile to
> create a new printer from ground up for this new version. If anyone is
> interested in doing the work, I'm sure there are users that are willing
> to throw some $s at it.
>
> [0]
> https://groups.google.com/forum/?fromgroups=#!topic/simile-widgets/melJm_A8flA
> 
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil.
> --
>
>
> --
> WINDOWS 8 is here.
> Millions of people.  Your app in 30 days.
> Visit The Windows 8 Center at Sourceforge for all your go to resources.
> http://windows8center.sourceforge.net/
> join-generation-app-and-make-money-coding-fast/
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
WINDOWS 8 is here. 
Millions of people.  Your app in 30 days.
Visit The Windows 8 Center at Sourceforge for all your go to resources.
http://windows8center.sourceforge.net/
join-generation-app-and-make-money-coding-fast/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW 1.8 and SMW core extension compatibility

[Markus Krötzsch]

On 27/10/12 08:35, Nischay Nahata wrote:
>
>
> On Mon, Sep 17, 2012 at 7:55 PM, Yaron Koren  <mailto:ya...@wikiworks.com>> wrote:
>
> Hi Markus,
>
> I'm curious about what you wrote about adding custom tables for
> different properties - is there any documentation about it?
>
> Hi,
>
> Strangely enough I came by this thread now.
>
> There is some documentation about this but might be outdated a bit now
> (Markus did you change some code there too?). Anyway, you can get some
> idea from http://semantic-mediawiki.org/wiki/Fixed_properties of how to
> do this.

This documentation is still accurate. The only change was that fixed 
properties that are declared in LocalSettings should use their DB key 
("Some_property_name" instead of "Some property name"). The rest is 
unchanged.

>
> Note that all the data of such a property would then be stored and
> looked up from the new table. So all data from the older table the
> property was using has to be migrated to this one now.

Right now this can be done by refreshing the data. We could add faster, 
more direct ways to do this in the future.

Markus

>
> As for the current code (at least in my extensions) that queries the
> SMW database tables directly, here's an explanation, in order from
> least to most challenging to "fix":
>
> - Semantic Internal Objects stores its own data in the tables; as
> you note, this can probably be changed to use the new code for
> handlng subobjects.
>
> - Semantic Forms does direct querying in order to autocomplete on
> substrings, in one function,
> SFAutocompleteAPI::getAllValuesForProperty(). You can see it here:
>
> 
> https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/SemanticForms.git;a=blob;f=includes/SF_AutocompleteAPI.php
>
> There's nothing in the SMW API that allows for substring matching
> the way SF does it (matching on the beginning of any word in the
> text); so something would have to be added.
>
> - Semantic Drilldown does direct querying because it needs to
> generate temporary tables, in order to improve performance: every
> time Special:BrowseData is accessed, temporary tables are created
> that hold the set of pages that match the current selected set of
> filters, and then all the additional numbers and values on the pages
> are generated off of those temp tables. To get that functionality
> into SMW would require, I think, some serious additions to the code;
> and I don't know if it would be worth the effort.
>
> (By the way, on that note - it would be great if SD could also
> handle cases where an RDF triplestore is used to hold SMW's data,
> but I think the lack of support for temporary tables in triplestores
> makes that impossible, regardless of whether it's SD or SMW doing
> the work of filtering. If anyone knows differently about that,
> though, please let me know.)
>
> -Yaron
>
>
> On Mon, Sep 17, 2012 at 6:06 AM, Markus Krötzsch
>  <mailto:mar...@semantic-mediawiki.org>> wrote:
>
> On 16/09/12 21:28, Yaron Koren wrote:
>
> Hi James,
>
> You make a valid point about the importance of SF and SD
> supporting
> SQLStore3 in the near future. I plan to add support for the
> new database
> structure to those extensions, as well as to Semantic
> Internal Objects,
> which also has hardcoded calls to the SMW tables - ideally
> that can
> happen in the next few weeks. I'm planning to do that by
> just adding
> "if" statements to the relevant parts of the code, so that
> the SQL used
> changes depending on which SQL store is being used. From my
> understanding of the new DB structure, it shouldn't be that
> hard to do.
>
>
> Yes and no. Regarding SIO you should be able to use SMW's API as
> discussed in earlier emails, so no table-related issues there.
> For SF, I am surprised that there is SQL-specific code at all.
> Some of the current store API functions are only provided for
> the use by SF, so if the kind of method that you need is not too
> hard to support, we could maybe do the same there.
>
> SD will be more difficult to upgrade, I think (without knowing
> the details of its code). One thing to know about the new store
> is that it allows users to configure the tables that will be
> used. The individual 

Re: [SMW-devel] Virtuoso and special characters

[Markus Krötzsch]

Hi,

this problem is not know (to me). A recent bugfix (bug 37575) corrected 
an encoding issue (char encoding not sent when calling RDF store; could 
lead to misunderstandings on the side of the store). You can try if this 
fixes your problem. Testing this can be done by replacing a single file 
in SMW 1.7; see the link to gerrit in the bug report for details.

This may or may not help you. To further analyse the problem, make sure 
that Virtuoso really contains the triple you expect. It could also be 
that the error is already introduced when writing the data. If the 
reading is the problem, it needs to be debugged further. You can do this 
by looking at the SPARQL query code (esp. the one that reads SPARQL 
result sets) and inserting print statements to view the raw data coming 
from the store, and the raw data extracted before turned into SMW 
objects. This should clarify where information is lost.

Good luck,

Markus

On 23/10/12 02:43, Danilo da Rosa wrote:
> Hello, we are having some trouble developing a MW extension, using SMW
> with Virtuoso, when having string objects with special characters in the
> triple store.
>
> When making a SPARQL query, through the “SMWSparqlDatabase::doquery
> ($sparql)” method, if the triple object has special characters, such as
> accents (i.e. á) or “ñ”, the result does not shows all the string, it
> shows only de part after the special char.
>
> For example, having the following triple in Virtuoso:
>
> http://127.0.0.1/id/MNunez http://127.0.0.1/id/Atribuoto:Nombre
> "Marcos Nuñez"
>
> And making this query, using the doQuery method:
>
> DEFINE input:inference 
> DEFINE get:soft "replacing"
> DEFINE input:same-as "yes"
> PREFIX rdf:
> select ?o where {
> {   ?o }
>
>   When we print the result, we get:
>
> ñez
>
> We would like to know if this is a known error, and if there is any
> workaround.
>
> Thanks in advance,
>
> Marcelo, Pablo y Danilo
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] database tables

[Markus Krötzsch]

On 22/10/12 20:57, Mauricio Etchevest wrote:
> Hi,
>
> Thanks for ansering.
>
> It's just for documentation, I´m developing an extension of SMW for my
> thesis.

Ok, feel free to ask more specific questions if you have them.

We are always interested in related works, so don't be shy to announce 
your prototype/thesis to the list when you have something published.

Where are you writing your thesis?

Markus

>
>
>
> On Mon, Oct 22, 2012 at 4:47 PM, Markus Krötzsch
> mailto:mar...@semantic-mediawiki.org>>
> wrote:
>
> On 22/10/12 19:55, Mauricio Etchevest wrote:
>
> Hi!
>
> I´m looking for documentation about how SMW store the data. How SMW
> store triples ?
>
> I search for documentation of the database tables and I found this:
> http://semantic-mediawiki.org/__wiki/Database_Tables
> <http://semantic-mediawiki.org/wiki/Database_Tables>
>
> This is all the documentation that we have?
>
>
> In essence, SMW uses a combination of a single-table and a property
> table approach, combined with a dictionary table to shorten page
> names. No compression mechanisms are applied to data values besides
> this dictionary. A lot of development is going on about this
> recently, so there is no accurate documentation of what we will soon
> have.
>
> May I ask why you are interested?
>
> Markus
>
>
>
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] database tables

[Markus Krötzsch]

On 22/10/12 19:55, Mauricio Etchevest wrote:
> Hi!
>
> I´m looking for documentation about how SMW store the data. How SMW
> store triples ?
>
> I search for documentation of the database tables and I found this:
> http://semantic-mediawiki.org/wiki/Database_Tables
>
> This is all the documentation that we have?

In essence, SMW uses a combination of a single-table and a property 
table approach, combined with a dictionary table to shorten page names. 
No compression mechanisms are applied to data values besides this 
dictionary. A lot of development is going on about this recently, so 
there is no accurate documentation of what we will soon have.

May I ask why you are interested?

Markus



--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Help translating SMW extensions

[Markus Krötzsch]

On 21/10/12 11:13, Federico Leva wrote:
> Hello,
>
> Markus Krötzsch, 19/10/2012 08:07:
>> Great, thanks a lot. I hope Nemo (Federico; CC) has seen this too.
>
> Yes, I've seen it and I've just started assigning requests to Karsten.
> Note, however, that in general they are not only about documentation,
> and I don't know if Karsten is also willing to commit patches for
> gender, grammar, plural support and so on when needed.

Requests that need changes to the code should be submitted to bugzilla 
as usual. Anybody can take any bug from there and submit a patch (in the 
best case directly through gerrit). Translation-related bugs do not 
require a special handling here.

In general, we can always use help in monitoring/processing input from 
bugzilla. This does not even require patches -- assessing, verifying, 
and classifying bugs would already help. Even filtering out the 
invalid/worksforme bugs would make a big contribution. But I18N requests 
are not special here.

Markus



>>>
>>> My recommendation to coders is to incluce message documentation right
>>> away. In fact this already done in the
>>> meantime in most cases.
>>
>> Yes, this is also important for other developers who may need to decide
>> if a message is appropriate for new code or not.
>


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Is smw-import-URI that corresponds to local page available through parser function?

[Markus Krötzsch]

On 18/10/12 23:02, Gregor Hagedorn wrote:
> SMW can be supplied with Mediawiki:SMW import pages that define that a
> local page has a corresponding global URI. RDF export uses this
> information.
>
> Is that information available to be evaluated and displayed from
> inside the wiki (especially through a parser function)? We have not
> yet found a way to achieve this.

No, this information is not available. I fact, the whole system of 
vocabulary import is in need of revision. It is very old and not as 
powerful/convenient as it should be.

Another major issue is that the mapping is non-invertible by design. So 
one cannot find the wiki property/category/page that belongs to some 
external URI used in vocabulary import. One cannot even ensure that this 
mapping is injective (many wiki entities can have the same external 
URI). This is problematic if RDF store backends are used for query 
answering: it is impossible to translate imported URIs back into wiki 
pages for displaying #ask results.

The whole system could use some redesign.

Markus


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] [Semediawiki-user] Help translating SMW extensions

[Markus Krötzsch]

Hi Karsten,

On 18/10/12 11:58, [[kgh]] wrote:
> Heiya Markus, heiya to all of you,
>
> when it comes to missing message documentation (/qqq) I offer to be the 
> person for first level support on
> bugzilla as well as on tw.n. As a matter of fact I have helped in this 
> context in the past already for SWM
> [1]. Thus bugs concerning missing /qqq may be assingned to me directly. I 
> just was not aware of this one
> (36607) and partially fixed it without knowing about it. The same applies to 
> tw.n support requests. You may
> use me as the main support contact for semantic extensions there, too.

Great, thanks a lot. I hope Nemo (Federico; CC) has seen this too.

>
> My recommendation to coders is to incluce message documentation right away. 
> In fact this already done in the
> meantime in most cases.

Yes, this is also important for other developers who may need to decide 
if a message is appropriate for new code or not.

Cheers,

Markus

>
> Cheers Karsten
>
> PS @Markus Please forward this message to the dev-list. Thank you.

The dev list was already in your CC, or do your messages get filtered? 
Anyway, this reply will reach both lists now.

>
> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=36608
>
>
> Am 18.10.2012 10:06, schrieb Markus Krötzsch:
>> Dear SMW users & developers,
>>
>> we enjoy an awesome internationalisation support from translatewiki.net,
>> which ceaselessly translates MW extensions (and more) into a huge number
>> of languages.
>>
>> To work smoothly, this also needs some help from the user/developer
>> community, since translators are sometimes not sure about how a message
>> is used (e.g. the word "view" could mean many things, and needs to be
>> translated differently in each case). If there is a problem, "Open
>> Requests" are created [1] (supported by SMW :-). Currently, there are 5
>> open requests related to SMW, Validator, SRF, and Semantic Expressivity
>> [2] which could use some looking into.
>>
>> More generally, the translatewiki.net community could always use the
>> help of users who are familiar with the software and can clarify the
>> intended meaning of a message. Various projects are lacking message
>> documentation (language code "qqq"), e.g. SRF [3]. This is something
>> where users who are familiar with the software could help by writing
>> short documentations. This can be done directly on translatewiki.net,
>> without any developer access to the code.
>>
>> If you would like to help out, please have a look at translatewiki.net.
>> They have a lot of documentation on how everything works (it is usually
>> quite easy). Part of the task is to chase the right people, so don't
>> hesitate to email the developer if something is unclear. I am sure that
>> some of you are already working with translatewiki anyway. Yet, the
>> folks at translatewiki are often not sure whom to ask about certain
>> issues -- if you think you could manage this a bit (i.e., become the
>> main contact for translating S* extensions) then you are most welcome to
>> let us know.
>>
>> Cheers,
>>
>> Markus
>>
>>
>> [1] https://translatewiki.net/wiki/Support/Open_requests
>> [2]
>> <https://translatewiki.net/wiki/Thread:Support/About_MediaWiki:Semex-shortquery-hover-loading-failed/en>
>> <https://translatewiki.net/wiki/Thread:Suppor/About_MediaWiki:Validator-listerrors-errors/en>
>> <https://translatewiki.net/wiki/Thread:Suppor/About_MediaWiki:Validator-type-title/en>
>> <https://translatewiki.net/wiki/Thread:Support/MediaWiki:Srf_paramdesc_views>
>> <https://translatewiki.net/wiki/Thread:Support/About_MediaWiki:Smw-ask-otheroptions-info/en>
>>
>> [3] https://bugzilla.wikimedia.org/show_bug.cgi?id=36607
>>
>> --
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_sfd2d_oct
>> ___
>> Semediawiki-user mailing list
>> semediawiki-u...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>>
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
> ___
> Semediawiki-user mailing list
>

[SMW-devel] Code documentation online again

[Markus Krötzsch]

Dear developers,

after a little downtime due to our server removal, the SMW&friends code 
documentation is live again:

http://semantic-mediawiki.org/doc/

The docs should be updated once a day. Please let me know if any other 
extension should be included there.

Cheers,

Markus

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] Help translating SMW extensions

[Markus Krötzsch]

Dear SMW users & developers,

we enjoy an awesome internationalisation support from translatewiki.net, 
which ceaselessly translates MW extensions (and more) into a huge number 
of languages.

To work smoothly, this also needs some help from the user/developer 
community, since translators are sometimes not sure about how a message 
is used (e.g. the word "view" could mean many things, and needs to be 
translated differently in each case). If there is a problem, "Open 
Requests" are created [1] (supported by SMW :-). Currently, there are 5 
open requests related to SMW, Validator, SRF, and Semantic Expressivity 
[2] which could use some looking into.

More generally, the translatewiki.net community could always use the 
help of users who are familiar with the software and can clarify the 
intended meaning of a message. Various projects are lacking message 
documentation (language code "qqq"), e.g. SRF [3]. This is something 
where users who are familiar with the software could help by writing 
short documentations. This can be done directly on translatewiki.net, 
without any developer access to the code.

If you would like to help out, please have a look at translatewiki.net. 
They have a lot of documentation on how everything works (it is usually 
quite easy). Part of the task is to chase the right people, so don't 
hesitate to email the developer if something is unclear. I am sure that 
some of you are already working with translatewiki anyway. Yet, the 
folks at translatewiki are often not sure whom to ask about certain 
issues -- if you think you could manage this a bit (i.e., become the 
main contact for translating S* extensions) then you are most welcome to 
let us know.

Cheers,

Markus


[1] https://translatewiki.net/wiki/Support/Open_requests
[2]





 

[3] https://bugzilla.wikimedia.org/show_bug.cgi?id=36607

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] [News] SMWCon Fall 2012 Program

[Markus Krötzsch]

Dear all,

the program for the upcoming SMWCon in Cologne is becoming more and more 
stable [1]. Most talks should be at their almost final location now. 
There are quite a few highlights that are worth mentioning:

* We have two exciting keynote talks by Denny Vrandecic (Wikimedia 
Germany e.V.) and Peter Haase (fluidOps):

Denny will introduce Wikidata, the next big thing for Wikipedia, and the 
underlying software Wikibase. The co-operation of SMW and Wikidata will 
be an important topic of this SMWCon.

Peter will introduce the Information Workbench, a semantic knowledge 
management solution by fluidOps. For the first time, SMWCon will include 
a number of talks on related systems that are not SMW. Other highlights 
in this category include OntoWiki, BlueSpice, SlideWiki, and the 
Drupal-based Planetary System. I am sure that it will be insightful and 
inspiring to exchange experiences with these projects.

* We'll have a number of practical experience talks. I am particularly 
looking forward to the insights of Wikia Inc., presented by Krzysztof 
Krzyżaniak (eloy).

* Joel Natividad will join us live from NY to report about his 
award-winning sites and new smart city projects.

* And of course there will be plenty of updates on SMW and its old and 
new extensions, including a number of presentations about using SMW in 
completely new ways.

The tutorial day will leave more space for discussions and practical 
work, especially to discuss problems and ideas with the developers (as 
usual, we will have a very high concentration of those). As a special 
"non-semantic" tutorial, Yury Katkov will share his first-hand 
experience in fighting spam on semanticweb.org and 
semantic-mediawiki.org, a topic that concerns many public SMW sites.

Cheers,

Markus

[1] http://semantic-mediawiki.org/wiki/SMWCon_Fall_2012/Agenda

--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Strange handling for special properties in SMWSQLStore3 DB design?

[Markus Krötzsch]

On 10/10/12 14:25, Yaron Koren wrote:
> Hi Markus,
>
> Thanks for the additional clarifications. I now understand, at the very
> least, why it helps for "Modification date" to have its own table; and
> more generally, I get the design rationale. I think now that it makes
> sense for Semantic Forms to switch to the page_props table, and parser
> functions, to store its form-linking information; and if that's what it
> takes for SMW to not have tables for SF properties, then it's all the
> more reason to do it. :)

If you don't want tables for the SF properties, then we can also do this 
without you moving to page props. But I don't think that this would be 
better, or at least we should analyse the situation based on actual DB 
load and not based on perceived oddness level ;-).

>
> By the way, I don't think additionally storing properties like "Has
> default form" would be that helpful - I don't believe people query on
> it, or look it up in Special:Browse, unless something goes wrong; and if
> parser functions like #default_form were to be used instead, it would
> probably be easier to know right away if there was a problem.

Sure, you know the SF users best.

>
> (Also, nobody should be concerned that support for "Has default form" is
> going away - if/when this change is made, I'm sure there will be
> backward compatibility preserved for a long time.)
>
> Anyway, hopefully this is enough to convince you and the other relevant
> people to take out the SF database tables - that was the part of the new
> design that felt the oddest to me.

There are various levels of special handling for the SF properties right 
now, all of them hardcoded. You should not worry about it too much.

Removing the special properties would make the form properties normal 
page-type properties; I am not sure that this will have a good impact on 
performance. The original reason for SMW to introduce special handling 
for SF properties was that SF makes very frequent requests to these 
properties. As long as this is still the case, at least some special 
handling (fixed property ID, fixed property type) should be preserved. 
Whether it is better to have many tables or one for the properties 
depends on how often you query each of the properties, how many there 
are (now we have two), and also how many other page-type property values 
pages typically have. I suggest to simply ignore the issue for now and 
to remove all mentioning of SF properties as soon as SF has no such 
properties any more.

Markus


>
> On Wed, Oct 10, 2012 at 6:44 AM, Markus Krötzsch
> mailto:mar...@semantic-mediawiki.org>>
> wrote:
>
> On 09/10/12 18:12, Yaron Koren wrote:
>
> Hi,
>
> Nischay - thanks for that set of links. I don't think any of
> them cover
> the thinking behind the design decisions, but they're still all
> good to
> know about.
>
> Markus - thanks for all the clarifications. I was sure that the
> extra
> tables were done to improve reading, but it sounds like they're
> mostly
> there to improve writing.
>
>
> I would not say that either of these is more important. Reading and
> writing are both relevant, and the new design should improve both
> activities in various situations.
>
>
> Is improving write performance a big deal,
> though? I'm no performance expert, but I would think that on an
> average
> wiki, writes are a fairly unimportant factor in the overall
> performance
> of the site. Is that incorrect?
>
>
> Current versions of SMW can cause a very high write load (writing on
> every edit, even if no data changed). This was an issue for Wikia
> and for anyone who is running MW on SSD drives. Just checking if
> some data has changed is not a solution, since the modification date
> always changes. Having more tables allows for a more fine-grained
> write control.
>
>
>
> I didn't mean that the number of tables (around 30 in
> SMWSQLStore3) will
> pose a technical challenge, just that it looks cluttered - and
> may pose
> challenges for administrators, and developers, in maintaining them.
>
>
> Not sure what kind of administration could be necessary. In general,
> users and administrators should not touch the tables.
>
>
>
> I'm still thinking about MediaWiki's page_props table - both as
> a design
> example, and as a tool that SMW and related extensions can use. As a
> design example: MediaWiki stores many different types of values
>

Re: [SMW-devel] [Semediawiki-user] [News] Google, Microsoft, Facebook And Others Launch SMW site

[Markus Krötzsch]

On 09/10/12 16:01, Krabina Bernhard wrote:
> this will be the SMW of the year, then ;-)

Well, we will have to see if interesting uses of properties happen there 
(I did not investigate the content yet). For now, the case mainly 
illustrates that SMW is among the basic extensions to start new MW 
projects with.

Markus

> - Ursprüngliche Mail -
>> Dear all,
>>
>> as TechCrunch reports [1]:
>>
>> "
>> A number of leading browser vendors and other tech companies,
>> including
>> Microsoft, Google, Apple, Adobe, Facebook, HP, Nokia, Mozilla, Opera
>> and
>> the W3C, just announced the launch of the Web Platform Docs project
>> at
>> WebPlatform.org. The project aims to create “a new, authoritative
>> open
>> web standards documentation site,” says Opera Software. The wiki-like
>> site, says Opera, wants to ensure that developers can easily find
>> “accurate, quality information on all the latest HTML5, CSS4 and
>> other
>> standards features across the multitude of available web-based
>> resources.”
>> "
>>
>> And guess what software they are using ... [2]
>>
>> Cheers,
>>
>> Markus
>>
>> [1]
>> http://techcrunch.com/2012/10/08/google-microsoft-facebook-and-others-launch-web-platform-docs-a-web-standards-documentation-site/
>> [2] http://docs.webplatform.org/wiki/Special:Version
>>
>>
>> --
>> Don't let slow site performance ruin your business. Deploy New Relic
>> APM
>> Deploy New Relic app performance management and know exactly
>> what is happening inside your Ruby, Python, PHP, Java, and .NET app
>> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
>> http://p.sf.net/sfu/newrelic-dev2dev
>> ___
>> Semediawiki-user mailing list
>> semediawiki-u...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>>
>


--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Strange handling for special properties in SMWSQLStore3 DB design?

[Markus Krötzsch]

On 09/10/12 18:12, Yaron Koren wrote:
> Hi,
>
> Nischay - thanks for that set of links. I don't think any of them cover
> the thinking behind the design decisions, but they're still all good to
> know about.
>
> Markus - thanks for all the clarifications. I was sure that the extra
> tables were done to improve reading, but it sounds like they're mostly
> there to improve writing.

I would not say that either of these is more important. Reading and 
writing are both relevant, and the new design should improve both 
activities in various situations.

> Is improving write performance a big deal,
> though? I'm no performance expert, but I would think that on an average
> wiki, writes are a fairly unimportant factor in the overall performance
> of the site. Is that incorrect?

Current versions of SMW can cause a very high write load (writing on 
every edit, even if no data changed). This was an issue for Wikia and 
for anyone who is running MW on SSD drives. Just checking if some data 
has changed is not a solution, since the modification date always 
changes. Having more tables allows for a more fine-grained write control.

>
> I didn't mean that the number of tables (around 30 in SMWSQLStore3) will
> pose a technical challenge, just that it looks cluttered - and may pose
> challenges for administrators, and developers, in maintaining them.

Not sure what kind of administration could be necessary. In general, 
users and administrators should not touch the tables.

>
> I'm still thinking about MediaWiki's page_props table - both as a design
> example, and as a tool that SMW and related extensions can use. As a
> design example: MediaWiki stores many different types of values in the
> page_props table. Table writes (and reads) could probably be a little
> faster if there were separate tables for different types of page
> properties, but the MediaWiki developers decided to just have one big
> table for everything.
>
> And as a tool, page_props could potentially be helpful for SMW, SF and
> others. SF's special properties, like "Has default form", don't need to
> be queried alongside real semantic properties - and actually, I doubt
> they're queried often at all, other than by SF itself. SF's information
> could be stored in the page_props table, instead of via SMW - and then,
> instead of an SMW tag, you could just add something like this to a
> category, property or template page:
>
> {{#default_form:City}}
>
> If it reduces clutter, administrator confusion, and compatibility
> issues, without increasing read or write times, it might be worth it...

I don't see what you mean with clutter and administrator confusion. I 
have never known how many tables MW uses internally, nor have the tables 
in MW confused me so far. We never had compatibility issues between SF 
and SMW so far either.

But pageprops could still be a good idea for SF. It should definitely 
improve read performance. You can also store the data redundantly in SMW 
and in PageProps, so you get the best of both (fast access, #ask, 
Special:Browse, data export). If you do this, we would probably make the 
SF properties less special in our code, since they would not be needed 
so heavily.

Markus


--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] [News] Google, Microsoft, Facebook And Others Launch SMW site

[Markus Krötzsch]

Dear all,

as TechCrunch reports [1]:

"
A number of leading browser vendors and other tech companies, including 
Microsoft, Google, Apple, Adobe, Facebook, HP, Nokia, Mozilla, Opera and 
the W3C, just announced the launch of the Web Platform Docs project at 
WebPlatform.org. The project aims to create “a new, authoritative open 
web standards documentation site,” says Opera Software. The wiki-like 
site, says Opera, wants to ensure that developers can easily find 
“accurate, quality information on all the latest HTML5, CSS4 and other 
standards features across the multitude of available web-based resources.”
"

And guess what software they are using ... [2]

Cheers,

Markus

[1] 
http://techcrunch.com/2012/10/08/google-microsoft-facebook-and-others-launch-web-platform-docs-a-web-standards-documentation-site/
[2] http://docs.webplatform.org/wiki/Special:Version


--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Strange handling for special properties in SMWSQLStore3 DB design?

[Markus Krötzsch]

On 08/10/12 15:44, Yaron Koren wrote:
> Hi,
>
> I'm finally trying out SMWSQLStore3, and figuring out how to get my
> extensions compatible with it, so I'm looking at the new table structure
> for the first time.
>
> First of all, is there any documentation about the design decisions that
> went into this new structure? Because it could be that this issue has
> been answered already.

Hi Yaron,

I don't think we have a comprehensive documentation yet (Nischay may 
correct me though). You already have a good idea of certain things; I 
will try to further clarify this below (I think there are some 
misunderstandings).

>
> If not - basically, the way the new database structure seem to work is
> that there's a separate table for each special property.

That is not the case. You are right that the new structure supports 
tables that are used for one property only. This has some advantages:

* The property is determined from the table; no need to store the 
property in each row -- less storage space, smaller indexes, less memory
* Changes for that property can be written independently of changes for 
other properties. For example, modification date changes on each edit, 
yet we do not need to update other properties/tables on each edit -- 
reduced write activity
* The table is smaller and more specific than general-purpose 
property-value tables. This gives MySQL (or any DB) a better chance for 
guessing selectivity when doing join optimisations in queries. -- faster 
query execution

There is a limit on how many tables a DBMS is happy with, but 30 tables 
are not a problem. Reading all data for one subject can become slower if 
the data resides in many tables, but reading data for a particular 
property should be faster, esp. if the respective property is used a lot 
(small tables fit into memory).

Now to your original question: it is not the case that all special 
properties have their own tables. The two things are independent: 
special properties can be stored in the common catch-all tables and 
normal properties can have their own table. It is just that SMW by 
default provides special tables for its own special properties. This 
includes SF properties that are heavily used (this dependency between 
SMW and SF has been there in similar form for many versions; properties 
used by SF all the time [on every page build] always had their datatype 
hardcoded in SMW).

There is a mechanism for users to create tables for "important" 
properties at their own discretion. I am not sure if this is documented yet.

> "Creation date"
> and "Modification date" both have one, as do "Has improper value for",
> and probably some others. Interestingly, there are also tables for the
> Semantic Forms special properties "Has default form" and "Has alternate
> form".
>
> This strikes me as strange design, for a few reasons:
>
> - I assume that this is done to speed up querying; but, as far as I
> know, many of SMW's special properties - like "Creation date" and the
> rest - are rarely queried on.

See above. Querying is only one aspect. Putting rarely used/rarely 
updated data into its own table allows this data to be disregarded in 
many cases, thus taking load off the rest of the DB.

>
> - This could lead to an explosion of database tables - it looks like
> there are about 30 in the new structure, which some might consider an
> explosion already, and if there are a bunch more special properties
> added for metadata (like "last author", "first author", etc.) the number
> could just keep growing indefinitely.

30 should not be a problem for any DBMS, and tables are not added 
automatically when special properties are added.

>
> - It's understandable that Semantic Forms would get special handling,
> but still, having one extension handle things for another introduces
> dependency issues. What if Semantic Forms got other special properties?
> Or what if, say, "Has alternate form" were removed? It could potentially
> lead to compatibility problems.

We can think about how to reduce this dependency. SF could probably 
control the property tables it wants to use by itself. However, I would 
recommend having tables for all of its frequently used properties.

>
> - On that note, SF already has two special properties that don't have
> their own table - "Page has default form" and "Creates pages with form",
> both of which can apply to forms just like "Has default form" and "Has
> alternate form" do. Not that I'm suggesting that SMW should get two more
> tables for these, but on the other hand, I believe SF queries on these
> on a fairly regular basis.

Then it would probably be good to have extra tables.

>
> So what could be done instead? I can think of a few options:
>
> - Store all properties, special and otherwise, in the same set of
> tables, and make better use of indexing to speed up queries.

We already make better use of indexing anyway, to the extent that we 
know how ;-). But MySQL is mainly using table-based selectivity 

[SMW-devel] Fwd: [Wikitech-l] Call to eliminate sajax

[Markus Krötzsch]

Developers,

the below email is most relevant for developers of SF and SRF 
(slideshow). It may also be of general interest to developers using 
sajax in private code.

Cheers,

Markus


 Original Message 
Subject: [Wikitech-l] Call to eliminate sajax
Date: Wed, 03 Oct 2012 10:18:48 -0700
From: Daniel Friesen 
Reply-To: Wikimedia developers 
To: wikitec...@lists.wikimedia.org

sajax is an ancient ajax library, it's part of our legacy code. And it
still gets to sneak a license note into our README.

It's probably about time that we start making sure that code is ready for
the day it disappears. Just as code shouldn't be relying on bits and
pieces of wikibits.

Currently the only parts of core that depend on sajax are the legacy
mwsuggest and upload.js.
mwsuggest is going to disappear when Krinkle's work making simplesearch's
suggestions work universally is finished.
I'm not sure what's going on with upload.js.

The real problem however is extensions. For some reason it appears that we
STILL have extensions depending on sajax. And I'm not talking about
ancient extensions on the wiki or in svn. I only did an ack through stuff
that's currently in git.

So I welcome anyone who is interested in going through extension code and
eliminating the use of sajax in favor of jQuery.ajax and RL.


core/skins/common/ajax.js
3:window.sajax_debug_mode = false;
4:window.sajax_request_type = 'GET';
7: * if sajax_debug_mode is true, this function outputs given the message
into
8: * the element with id = sajax_debug; if no such element exists in the
document,
11:window.sajax_debug = function(text) {
12: if (!sajax_debug_mode) return false;
14: var e = document.getElementById( 'sajax_debug' );
18: e.className = 'sajax_debug';
19: e.id = 'sajax_debug';
41:window.sajax_init_object = function() {
42: sajax_debug( 'sajax_init_object() called..' );
61: sajax_debug( 'Could not create connection object.' );
77: *sajax_do_call( 'doFoo', [1, 2, 3], document.getElementById(
'showFoo' ) );
83:window.sajax_do_call = function(func_name, args, target) {
88: if ( sajax_request_type == 'GET' ) {
105:x = sajax_init_object();
112:x.open( sajax_request_type, uri, true );
119:if ( sajax_request_type == 'POST' ) {
130:sajax_debug( 'received (' + x.status + ' ' + x.statusText + ') 
' +
x.responseText );
153:alert( 'bad target for sajax_do_call: not a function or 
object: ' +
target );
157:sajax_debug( func_name + ' uri = ' + uri + ' / post = ' + post_data );
159:sajax_debug( func_name + ' waiting..' );
169:var request = sajax_init_object();

core/skins/common/mwsuggest.js
489:var xmlhttp = sajax_init_object();

core/skins/common/upload.js
99: if ( !ajaxUploadDestCheck || !sajax_init_object() ) return;
124:if ( !ajaxUploadDestCheck || !sajax_init_object() ) return;
133:if ( !ajaxUploadDestCheck || !sajax_init_object() ) return;
141:sajax_do_call( 'SpecialUpload::ajaxGetExistsWarning',
[this.nameToCheck],
287:var req = sajax_init_object();

extensions/CommunityVoice/Resources/CommunityVoice.js
100:var oldRequestType = sajax_request_type;
102:sajax_request_type = "POST";
104:sajax_do_call(
114:sajax_request_type = oldRequestType;

extensions/DonationInterface/modules/validate_input.js
14: sajax_do_call( 'efPayflowGatewayCheckSession', [], checkSession );

extensions/Drafts/Drafts.js
76: var oldRequestType = sajax_request_type;
78: sajax_request_type = 'POST';
80: sajax_do_call(
98: sajax_request_type = oldRequestType;

extensions/OnlineStatus/OnlineStatus.js
35: sajax_do_call( 'OnlineStatus::Ajax', ['get'], function( x ){
71: sajax_do_call( 'OnlineStatus::Ajax', ['set', status], function( x ){

extensions/ReaderFeedback/readerfeedback.js
36:  /*extern sajax_init_object, sajax_do_call */
92: sajax_do_call( "ReaderFeedbackPage::AjaxReview", args,
wgAjaxFeedback.processResult );

extensions/SecurePoll/resources/SecurePoll.js
95: sajax_do_call( 'wfSecurePollStrike', [ action, id, reason ],
processResult );

extensions/SemanticForms/includes/SF_FormUtils.php
450:function FCK_sajax(func_name, args, target) {
451:sajax_request_type = 'POST' ;
452:sajax_do_call(func_name, args, function (x) {
716:sajax_request_type = 'POST' ;
718:sajax_do_call('wfSajaxWikiToHTML', [SRCtextarea.value], 
function (
result ){
736:if (!oFCKeditor.ready) return false;//sajax_do_call in 
action
754:sajax_request_type = 'GET' ;
755:sajax_do_call( 'wfSajaxToggleFCKeditor', ['hide'], 
function(){} )
;   //remember closing in session

extensions/SemanticForms/libs/SF_ajax_form_preview.js
43: var aj = sajax_init_object();
44: var aj2 = sajax_init_object();

[SMW-devel] [News] Join us for SMWCon & present your work

[Markus Krötzsch]

Dear all,

SMWCon 2012 in Cologne is approaching quickly [1]. Hot topics this time 
will be Wikidata (and its relationship with SMW), performance and 
search, but also some very interesting new applications of SMW. We will 
also have a rare local concentration of SMW developers to discuss 
current developments with.

In the best tradition of SMWCons, we are still in the process of 
finalising the program (it will probably be really final only after the 
event). However, it is a good time *now* to announce your presentation 
plans to help planning. If you did not announce your talk on the SMWCon 
page yet, you can also drop us an email right away. We welcome 
contributions about all things SMW, including applications, extensions, 
experiences, and services (possibly commercial). There will be a 
lightning talk session for super short presentations.

Due to the generous support from our host GESIS, registration fees could 
be kept very low this time (note that a lot of food is included), and 
Cologne is very easy to reach (e.g., from Frankfurt airport). So there 
is really no reason for not joining us this time ;-)

Looking forward to seeing you in Cologne,

Markus

[1] http://semantic-mediawiki.org/wiki/SMWCon_Fall_2012


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] SMW 1.8 and SMW core extension compatibility

[Markus Krötzsch]

On 17/09/12 18:55, Yaron Koren wrote:
> Hi Markus,
>
> I'm curious about what you wrote about adding custom tables for
> different properties - is there any documentation about it?
>
> As for the current code (at least in my extensions) that queries the SMW
> database tables directly, here's an explanation, in order from least to
> most challenging to "fix":
>
> - Semantic Internal Objects stores its own data in the tables; as you
> note, this can probably be changed to use the new code for handlng
> subobjects.

Yes.

>
> - Semantic Forms does direct querying in order to autocomplete on
> substrings, in one function,
> SFAutocompleteAPI::getAllValuesForProperty(). You can see it here:
>
> https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/SemanticForms.git;a=blob;f=includes/SF_AutocompleteAPI.php
>
> There's nothing in the SMW API that allows for substring matching the
> way SF does it (matching on the beginning of any word in the text); so
> something would have to be added.

Yes, this is tricky, and I don't think we can properly support this with 
this storage backend. Ideally, one would use Lucene for this (this is 
what Wikimedia does on its projects). What SMW does support in all 
storage functions is substring matching from the start of the string. 
Would this be a possible fallback option to retain some functionality 
while no other solution is found?

Upcoming SMW versions should have hooks that allow extensions to listen 
to data changes. Through this, an extension could maintain its own 
tables (or other backend) to be used for special tasks such as auto 
completion. This would make this function independent of the storage 
backend, at the cost of using more storage. But different data access 
requirements should best be served by different data structures.

>
> - Semantic Drilldown does direct querying because it needs to generate
> temporary tables, in order to improve performance: every time
> Special:BrowseData is accessed, temporary tables are created that hold
> the set of pages that match the current selected set of filters, and
> then all the additional numbers and values on the pages are generated
> off of those temp tables. To get that functionality into SMW would
> require, I think, some serious additions to the code; and I don't know
> if it would be worth the effort.

Yes, this is tricky. Again, it might be better to maintain auxiliary 
data structures continuously based on watching SMW changes. Even with 
the temporary tables, SD is very slow right now due to the complicated 
queries it needs. Maybe one could reduce the effort by adding more 
(aggregate) information to the store that is updated on edit (relatively 
rare) instead of being updated on browsing (relatively frequent).

>
> (By the way, on that note - it would be great if SD could also handle
> cases where an RDF triplestore is used to hold SMW's data, but I think
> the lack of support for temporary tables in triplestores makes that
> impossible, regardless of whether it's SD or SMW doing the work of
> filtering. If anyone knows differently about that, though, please let me
> know.)

The current triple store support maintains all SQL tables as well. So 
the data can be taken from there in any case. The above approach (listen 
to changes to maintain your own tables) would also work in any case.

But I agree that all of this is not so easy to realise and not something 
that can be done immediately. Making a fast faceted browser (SD 2.0 ;-) 
would be a project on its own.

Markus

>
> -Yaron
>
>
> On Mon, Sep 17, 2012 at 6:06 AM, Markus Krötzsch
> mailto:mar...@semantic-mediawiki.org>>
> wrote:
>
> On 16/09/12 21:28, Yaron Koren wrote:
>
> Hi James,
>
> You make a valid point about the importance of SF and SD supporting
> SQLStore3 in the near future. I plan to add support for the new
> database
> structure to those extensions, as well as to Semantic Internal
> Objects,
> which also has hardcoded calls to the SMW tables - ideally that can
> happen in the next few weeks. I'm planning to do that by just adding
> "if" statements to the relevant parts of the code, so that the
> SQL used
> changes depending on which SQL store is being used. From my
> understanding of the new DB structure, it shouldn't be that hard
> to do.
>
>
> Yes and no. Regarding SIO you should be able to use SMW's API as
> discussed in earlier emails, so no table-related issues there. For
> SF, I am surprised that there is SQL-specific code at all. Some of
> the current store API functions are only provided for the use by SF,
> so if the k

Re: [SMW-devel] SMW 1.8 and SMW core extension compatibility

[Markus Krötzsch]

On 16/09/12 21:28, Yaron Koren wrote:
> Hi James,
>
> You make a valid point about the importance of SF and SD supporting
> SQLStore3 in the near future. I plan to add support for the new database
> structure to those extensions, as well as to Semantic Internal Objects,
> which also has hardcoded calls to the SMW tables - ideally that can
> happen in the next few weeks. I'm planning to do that by just adding
> "if" statements to the relevant parts of the code, so that the SQL used
> changes depending on which SQL store is being used. From my
> understanding of the new DB structure, it shouldn't be that hard to do.

Yes and no. Regarding SIO you should be able to use SMW's API as 
discussed in earlier emails, so no table-related issues there. For SF, I 
am surprised that there is SQL-specific code at all. Some of the current 
store API functions are only provided for the use by SF, so if the kind 
of method that you need is not too hard to support, we could maybe do 
the same there.

SD will be more difficult to upgrade, I think (without knowing the 
details of its code). One thing to know about the new store is that it 
allows users to configure the tables that will be used. The individual 
table layout is quite similar to what we already have, but there can be 
a lot more (smaller) tables. In particular, some (important) properties 
can get their own tables.

Cheers,

Markus

>
> By the way, some people might argue that these extensions shouldn't be
> making direct SQL calls at all, which is a valid point; but the problem
> is that, as far as I know, there's no other way at the moment to
> retrieve (or, in SIO's case, set) the necessary data. If someone wants
> to add such a capability to SMW for the use of one or more of these
> extensions, I'd be glad to discuss it.
>
> -Yaron
>
> On Fri, Sep 14, 2012 at 10:45 AM, James HK  > wrote:
>
> Hi,
>
> SMW 1.8beta has been released but some SMW core extensions (SF, SD
> etc.) using direct access to smw db tables that correspond to the
> SQLStore2 scheme[1]/[2] but are not compatible with the new SQLStore3
> scheme.
>
> In case of SF 2.5 one can use SMW 1.8/SQLStore2 but not SMW
> 1.8/SQLStore3 (because of the SQLStore2 specific db selects).
>
> How to resolve this situation that in both cases (SQLStore2/SQLStore3)
> the integration works without interference?
>
> I think this question needs some attention before releasing SMW 1.8
> because when people start using the SQLStore3 in combination with core
> extensions like Semantic Forms or Semantic Drilldown problems will
> occur at the current implementation state.
>
> [1]
> 
> https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/SemanticForms.git;a=blob;f=includes/SF_AutocompleteAPI.php;h=8edc80fbefe0915a75451c2a9a6befa4f8033a14;hb=b95c7db0e1ab010fdeb3e70ef0692b8df406713a#l157
>
> [2]
> 
> https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/SemanticDrilldown.git;a=blob;f=specials/SD_BrowseData.php;h=5f09df9f4259a598b6e23e86a9fc67f6eb7027c9;hb=HEAD#l266
>
> Cheers
>
> 
> --
> Got visibility?
> Most devs has no idea what their production app looks like.
> Find out how fast your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219671;13503038;y?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
>
>
> --
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] 4store issue

[Markus Krötzsch]

On 06/09/12 00:33, Samuel Rose wrote:
> using:
>
> MediaWiki   1.19.2
> Semantic Bundle (Version 20120327)
> 4store v1.1.4
>
> When I try to update or create a page with RDF data, 4store returns
> the following error:
>
> Sep  5 23:09:47 holocene 4store[3752]: update.c:80 error: syntax
> error, unexpected '{' at line 9 of operation 0

Could you provide a link to or the contents of the affected page? Maybe 
some property value or property name contains unusual characters?

Cheers,

Markus

> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] GSoC on performance... any results?

[Markus Krötzsch]

On 05/09/12 04:52, Laurent Alquier wrote:
> Hi
>
> Do anyone know if any results have been published about the kind of
> performance improvements we can expect from this work?

Many independent improvements have been done. Most notably regarding 
performance:

* significantly reduced SMW write activity (a major issue for some sites)
* overhauled special page computations to fix critical performance 
issues there (another major issue for some sites)
* more structured database layout (more smaller tables) that should 
improve MySQL performance in general (maybe including the part of #ask 
performance that is related to SQL query execution, but this is not known).

We are still looking into improved read performance and reduced reading 
activity (there is still a bug leading to too many repeated queries). 
The new semantic diff code will also allow more efficient/selective 
execution of other SMW operations, but this still needs to be implemented.

>
> I have been looking at the project blog and I can't help but noticing
> the lack of data - graphs, comparison before and after improvements,
> benchmarking... that sort of thing.

We have not focussed on experimental evaluation so far, mainly because 
there are still more improvements scheduled before release.

This will have to wait for a week since I am travelling a lot now (as 
usual, Nischay is ahead of me ;-).

Cheers,

Markus

>
> https://greensmw.wordpress.com/
>
> Looking forward to hearing more about this!
>
> Cheers
>
> --
> - Laurent Alquier
> http://www.linfa.net
>
>
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[SMW-devel] How to get property values from SMW (Was: The right type for storing IP addresses)

[Markus Krötzsch]

On 06/08/12 15:59, Yury Katkov wrote:
> Hi Jeroen!
>
> Each piece of programming documentation is as valuable as gold! Please do.
> I've spent about two days recently before I figured out these two
> lines that allow to want the value of the semantic property.
>
>   $params = array ("[[Pagename]]", "?ExampleProperty=", "mainlabel=-");
>   $result = SMWQueryProcessor::getResultFromFunctionParams( $params,
> SMW_OUTPUT_WIKI );

The above is a very complicated way to get the data you want 
(constructing a whole query with printouts to get a single data value). 
It's short, but it invokes all kinds of string-based parsing that may 
not be needed. Instead, you can do:

// Assuming that you have the names of page and property fixed:
// (all strings are DBKeys, that is, use '_' instead of ' ')
$propertyDi = new SMWDIProperty( 'ExampleProperty' );
$pageDi = new SMWDIWikiPage( 'Pagename', NS_MAIN, '' );
$valueDis = smwfGetStore()->getPropertyValues( $pageDi, $propertyDi );
// $valueDis now is an array of all property values, given as DI objects

// To create DV objects:
foreach ( $valueDis as $valueDi ) {
$valueDv = SMWDataValueFactory::newDataItemValue( $valueDi, $propertyDi 
);
// do something ...
}

If page or property name comes from user input, then you can do 
something like:

$propertyDv = SMWDataValueFactory::newTypeIdValue( '__pro', $propertyName );
$pageDv = SMWDataValueFactory::newTypeIdValue( '_wpg', $pageName );

and then call getDataItem() to get the DIs. The strings '__pro' and 
'_wpg' are the type ids that control how SMW will handle the string to 
create a suitable DV object. You can see all builtin type ids in 
SMWDatavalueFactory, if you want to know this for other datatypes as 
well. However, if you have "clean" internal values, you can always 
create DI objects directly, rather than parsing them from a string.

Please take advantage of the devel list for such questions :-)

Cheers,

Markus


>
> I'm pretty sure that making custom datatypes is pretty common task and
> not only I will find your docs useful.
> -
> Yury Katkov
>
>
>
>
> On Mon, Aug 6, 2012 at 5:47 PM, Jeroen De Dauw  wrote:
>>
>> Hey,
>>
>>> The sorting is not ideal, but of course there's no "IP address" type,
>> which is what would be required here.
>>
>> It's pretty trivial to create a DataValue for this, which is all that's
>> needed to have such a build in type. If anyone is interested in doing the
>> little programming work, I can write up some docs :)
>>
>> Cheers
>>
>> --
>> Jeroen De Dauw
>> http://www.bn2vs.com
>> Don't panic. Don't be evil.
>> --
>> --
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> ___
>> Semediawiki-user mailing list
>> semediawiki-u...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> ___
> Semediawiki-user mailing list
> semediawiki-u...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Triple Store URIs: index.php/Page or index.php?title=Page

[Markus Krötzsch]

On 01/08/12 00:57, Benedikt Kämpgen wrote:
> Hello,
>
> I have connected a wiki to an Open Virtuoso triple store as documented in [1].
>
> The wikis URIs have the form "...index.php?title=Page", e.g., [2].
>
> Yet, in the triple store, URIs are stored in the form "...index.php/Page", 
> e.g., [3], so that queries return empty results.
>
> The strange thing is, when I visit a page, do edit and save, then the proper 
> URIs [1] are stored.
>
> Purging SMW as described in [4] or via Special:SMWAdmin does not help and 
> might even be the reason for storing the wrong URIs in the first place.
>
> An Apache rewrite and usage of the $wgArticlePath setting of MW might be an 
> work around here, but the issue might still be a bug.
>
> Any help would be appreciated.

Hi Benedikt,

this is strange, because URIs are crated from URLs. I also have the 
index.php standard setting on my wiki and I do get URIs of the form

http://localhost/smwtestwiki/index.php?title=Special:ExportRDF/Test

At least this is what I see in the RDF export now. Do you have the same 
URIs in Special:ExportRDF? Do you have any setting for $wgArticlePath in 
your LocalSettings?

Best,

Markus

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Bug in concept-conditions ?

[Markus Krötzsch]

On 30/07/12 11:55, Roland Wohlfahrt wrote:
> Hi folks,
>
> I have 2 concepts in my semantic mediawiki:
>
> concept 1:
>
> [[Kategorie:Kontakt]] [[Position::Amtsleiter]] OR [[Kategorie:Kontakt]] 
> [[Position::Amtsleiterin]]
>
>
> concept 2:
>
> [[Kategorie:Kontakt]] [[Position::Amtsleiter||Amtsleiterin]]
>
>
> The resulting sets of these concepts *are different* because the latter
> one (concept 2) returns also articles that have "Amtsleiter" in another
> property. (not strictly using the property "Position").
>
> Bug? Or wrong query on my side?

This would be a bug. Can you file it? I think there was some similar bug 
some time ago, but it was never identified clearly what the reason was. 
The description you gave suggests that it might be as simple as a 
missing parentheses in SQL.

Cheers,

Markus

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query result caching and invalidation (Jeroen De Dauw)

[Markus Krötzsch]

On 27/07/12 05:12, jmccl...@hypergrove.com wrote:
> I'm sorry, I meant to say that Concepts cache queries, so it suggests
> there's a mechanism driving the creation of Concept pages. It'd be
> interesting to hear about that mechanism and whether it can be used with
> Concept pages in SMW.

Concepts have some similarities in that they are also conceived as 
"cached queries". The main difference to our current discussion is that 
we now want to update these caches automatically to ensure that they are 
always up-to-date. This will also require us to store more information 
about each query.

The concept-based caching, in contrast, is a manual approach where users 
create and update caches. It is intended for queries that are in general 
too slow to be computed at page render time (which would be necessary in 
all other cases, even if we maintain a cache of the results once they 
are computed).

As a side effect, the new solution would also keep track of all queries 
that are used on the wiki, which could be useful for other purposes 
(including performance analysis). Right now, there is no good way to 
find out which queries are used on a wiki.

Markus

> On 26.07.2012 14:10, Jeroen De Dauw wrote:
>
>> Hey,
>>
>> > Would you pleaes explain what is being gained by query caching?
>>
>> https://en.wikipedia.org/wiki/Cache_%28computing%29
>>
>> > It seems to me that simple transclusion of pages stored on wikidata
>> would be just as effective, would require no additional code.
>>
>> * This has nothing to do with Wikidata
>> * This has nothing to do with transclusion of pages
>> * This definitely requires code to work
>>
>> Cheers
>>
>> --
>> Jeroen De Dauw
>> http://www.bn2vs.com
>> Don't panic. Don't be evil.
>> --
>>
>
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> ___
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>



--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query result caching and invalidation

[Markus Krötzsch]

On 26/07/12 17:22, Jeroen De Dauw wrote:
> Hey,
>
>  > On the other hand, it would be even more useful to cache all results
> per (sub)query, ignoring the limit
>
> This can reduce computing overlapping results, but on the other hand is
> likely to compute results we'll never actually use. And it makes the
> implementation more complex. Since I'm not convinced the actual result
> would be better (I suspect that in fact it's be worse), I prefer to keep
> it simple for now. And if you have a case where the "store everything"
> approach really makes sense, you can always use a concept right?
>
>  > I was thinking of caching the query result only, not the printouts.
> One could cache a list of results, instead of caching all data needed to
> display the query result.
>
> Similar arguments apply here. Any query obtaining a single property
> would automatically fetch all properties for all matching objects. Again
> I don't think it's that much of an improvement. Esp considering the
> following:
>
>  > Having the lists of query results that are displayed in one query now
> could be useful for updating (if you have a data blob, you cannot check
> quickly for which queries a page occurs as a displayed result).
>
> Sure, I'd make it easier to figure this out. At least, if you go
> invalidate it whenever a single property changes. So now our query
> obtaining a single property does not only result into all properties
> getting obtained, but it'll also have it's cache invalidated whenever
> one of those other properties is changed.

Not necessarily. One can still store the printout properties and look at 
the diff to see if any of them was affected.

> This seems like something we
> really should avoid, so we'll have to hold into account the affected
> properties anyway, making the "just store all properties" approach not
> simpler to implement.
>

Not sure what "just store all properties" means. I was arguing for the 
opposite: not to store the properties again, since the printouts can 
easily be fetched from DB in the (relatively rare) cases that the parser 
cache needs to be rebuild. Mirroring all printout properties in the 
query cache would require more frequent updates to it and make it more 
specific to one single page.

But it does not matter much for now. The big issue with all of the query 
result caching is to limit the amount of cache invalidation that happens 
on updates. We need to think about how to get more specific information 
about queries than the properties that they refer to. Some wikis have 
thousands of pages with very similar queries, always using the same 
property (from a template), where each query has only a few results 
(referata gives a good example). A property-based cache invalidation 
would kill most of the query caches on almost every property edit (there 
are often just a handful of properties). Storing results for 
(sub)conditions as an exhaustive list could allow a much more 
fine-grained control of cache invalidation. The challenge is to keep 
these sets small. Maybe there are other approaches as well, such as 
singling out certain "selective" subqueries for this purpose.

Markus

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query result caching and invalidation

[Markus Krötzsch]

On 26/07/12 16:03, Jeroen De Dauw wrote:
> Hey,
>
>  > Various queries can have the same condition but different parameters
> (e.g., showing it in a table vs. showing it in a list).We should only
> have one cache for two queries that have the same results.
>
> Fully agree with this. I'm proposing to cache the result of a queries,
> not the visualizations of them. So the cache key would consist of the
> conditions, the printouts, offset, limit, sort, and perhaps some other
> relevant parameter I'm forgetting.

I was thinking of caching the query result only, not the printouts. One 
could cache a list of results, instead of caching all data needed to 
display the query result. The latter would be more like a blob of data, 
the former would be more like a category table ("page|query"). Both 
would have their use, I suppose, but the query display as a whole is 
already cached in the parser cache, so it might not be needed to cache 
the whole thing again. Having the lists of query results that are
displayed in one query now could be useful for updating (if you have a 
data blob, you cannot check quickly for which queries a page occurs as a 
displayed result). On the other hand, it would be even more useful to 
cache all results per (sub)query, ignoring the limit ... not sure if we 
could afford this [would not be necessary for subqueries that are 
already stored in such a literal way, like namespace or category queries].

>
>  > We can still have a query table (it would be interesting to store
> information about individual queries beyond the condition they have).
>
> You need this either with this approach or the one you're arguing
> against no?

You probably want it in any case. That's what I meant.

Markus

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

Re: [SMW-devel] Query result caching and invalidation

[Markus Krötzsch]

On 26/07/12 15:39, Jeroen De Dauw wrote:
> Hey,
>
> Nischay, Markus and me where discussing how to implement caching for ask
> queries and inadvertently ended up discussing the whole query
> invalidation project again. Since this fits in with Nischay his project
> and is something I also want to poke at since we'll have to implement
> something similar in Wikidata I decided to write up my current thoughts
> on how to implement this.
>
>
> I propose having a table "queries" where each row an identifier for a
> query (for instance a hash of the conditions, printouts and relevant
> params). Entries would be added on page save in case they are not there
> yet. Each entry can contain the computed results for the query. There
> would also be a table to map each query to the pages on which it's used.
> Flow would look like this:
>
>
> * People use a single query on multiple pages, first usage inserts a new
> entry in queries with the freshly obtained result
>
> * Successive usages just get the result from the cache in the query table
>
> * When someone changes data, we figure out what queries can be affected
> and remove their cache, plus invalidate the parser,html,whatever caches
> of all pages containing any of the queries that had their cache removed
>
> * On next view of such a page SMW find an empty cache for the query and
> recomputes it
>
> Note: we would not necessarily need to wait for people to view a page to
> have the cache rebuild (both the query cache and the page specific
> caches). We could create jobs to do this, so that on the next view of
> the page, it's there immediately. This only makes sense for wikis where
> most pages get visited often though, since else you might be doing a lot
> of work for nothing.
>
>
> The only difficult problem left to solve here is how to best figure out
> which queries have changed (or could have changed), but this does not
> appear to affect the rest of the design.
>
> Anyone objections against such an approach or suggestions of any sort?

I think we should cache results per query condition rather than per 
query. Various queries can have the same condition but different 
parameters (e.g., showing it in a table vs. showing it in a list). We 
should only have one cache for two queries that have the same results. 
We can still have a query table (it would be interesting to store 
information about individual queries beyond the condition they have).

For change propagation and cache update, it might help to decompose the 
query condition (along its PHP object structure as an SMWDescription). 
Every single description part is relatively simple, yet precise enough 
to check if it was affected by a change. In this way, changes could be 
pushed through the description objects. This would also allow much 
stronger reuse of subconditions that occur in many query descriptions. 
The details need to be worked out ...

Markus

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel