Re: [SMW-devel] Semantic mediawiki and map data
I seem to have isolated the issue to the method getProperyValues in the class SqlStubSemanticData. Here it seems that there is some kind of mixup going on when i have defined a container to hold coordinates. According to the code it seems that it tries to fetch a given array entry with the key of the property in mStubPropVals. So if i have defined the property "Polygon" that is of type _gpo (Graphical Polygon), this type holds a list of _geo properties which is also defined as a semantic property "Coordinates" However, when it this instance of Property is loaded in SqlStubSemanticData, i can see that the mStubPropVals is populated with a key of "Coordinates" when it is looking for a key "Polygon". Any ideas on what is going on here? Can someone please elaborate the usage of SqlStubSemanticData and how it is implemented in the project? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
I've been staring at the code now for the past few hours, and i simply cant figure out why my data isn't retrieved. Unless i can get some more support on this then i'm afraid i will have to seize my plans on extending SMW with polygons, circles, rectangles, and lines datastructures. But i reckon if i can just get polygons to work 100%, then it would be fairly easy for me to implement the rest. Polygons and lines have the same structure, so they should be implemented the same way. Circles and rectangles have two parameters instead of an unknown number, so they would act much in the same way as todays geographical coordinate. So i guess the question is, can either of you, or maybe both (Jeroen and Markus) take a look at the code under the branch polygons, and see what you make of it. Is there something missing? Am i using the container structure in a way that is completely and utterly wrong? Have i managed to register all the hooks that i need so that datavalues are loaded properly? Kim On Tue, Jun 12, 2012 at 10:13 AM, Kim Eik wrote: > Hi again, sorry for the long wait. I have been working on some slightly > easier tasks for the time being, but have now returned to this :) So, i'm > still having issues where when querying the data from the #ask parser is > not showing any data (empty table cell). Please take a look at the polygons > branch on SMW and SM if you have the time. But anyways, back to the > response i recieved from you. > > >> Did you register your new datatype (DV class) to use containers? This is >> done in the hook smwInitDatatypes; see SMW_DatavalueFactory for >> docuemntation (it's similar to the property registration, using >> registerDatatype and registerDatatypeAlias). Without this, SMW won't know >> that you need a container and it will just give you the subobject (wikipage >> DI) when retireving data from the store; this would then be rejected by >> your DV, and no values would be created. >> > > Yes i believe i have so, i have registered the datavalue through calling: > > > SMWDataValueFactory::registerDatatype( '_gpo', 'SMGeoPolygonValue', > SMWDataItem::TYPE_GEOPOL ); > > > in the SemanticMaps extension which is in a method that is called on the > smwInitDatatypes hook. > > In the SemanticMediaWiki extension i have registered through adding: > > '_gpo' => SMWDataItem::TYPE_GEOPOL, // Geographical polygon > > to the self::$mTypeDataItemIds array in the function initDataTpes in > SMWDataValueFactory. > > Is there anyway i can verify if i have registered this datavalue properly? > for example adding a breakpoint to a method that SMW would call in the > datavalue? > > >> >> >> Now for the first issue (which may not be your problem yet, but will >> probably come up later). Container-type data values need to be queried >> differently. Every datavalue has its own code for deciding what to query >> when a user enters some string in #ask. This is implemented in the >> SMWDataValue's method getQueryDescription(). The default handling is to >> interpret initial comparators (<, >, ...) first, and then parse the rest >> like a user input. The result is used for querying. This does not work for >> containers, since, IIRC, SMW does not currently support the use of >> container DIs in queries (i.e., in the PHP level description of queries). I >> might be wrong here. >> > > So far i have just added a stub method getQueryDescription in the > datavalue class, however it's not being called (yet), however i'm guessing > that is because of my issue above. > > >> >> The pedestrian solution for now is to implement something like in >> SMW_DV_Record.php. The code there has two modes during parsing: either >> construct a DIContainer or construct a query description. This is actually >> needed there because Records also support comparators (<, >, ...) for >> *each* of their component values in queries; so the parsing is sightly >> different (it must allow comparators in front of each value, not just at >> the very beginning). If this is not desired, then one could also have the >> same parsing and just turn the container into a query (this might already >> have code somewhere; not sure). The only thing to do differently would be >> to use an anonymous subject for the Container in this case -- otherwise the >> query will search for exactly the subject name that you have given, which >> is not correct. >> > > I will have to get back to you on this one when i get that far. > > >> >> This thread is getting into the most comprehensive documentation of SMW >> internals ever written ;-) >> >> Markus >> > > Cheers. > Kim > -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/501
Re: [SMW-devel] Semantic mediawiki and map data
Hi again, sorry for the long wait. I have been working on some slightly easier tasks for the time being, but have now returned to this :) So, i'm still having issues where when querying the data from the #ask parser is not showing any data (empty table cell). Please take a look at the polygons branch on SMW and SM if you have the time. But anyways, back to the response i recieved from you. > Did you register your new datatype (DV class) to use containers? This is > done in the hook smwInitDatatypes; see SMW_DatavalueFactory for > docuemntation (it's similar to the property registration, using > registerDatatype and registerDatatypeAlias). Without this, SMW won't know > that you need a container and it will just give you the subobject (wikipage > DI) when retireving data from the store; this would then be rejected by > your DV, and no values would be created. > Yes i believe i have so, i have registered the datavalue through calling: SMWDataValueFactory::registerDatatype( '_gpo', 'SMGeoPolygonValue', SMWDataItem::TYPE_GEOPOL ); in the SemanticMaps extension which is in a method that is called on the smwInitDatatypes hook. In the SemanticMediaWiki extension i have registered through adding: '_gpo' => SMWDataItem::TYPE_GEOPOL, // Geographical polygon to the self::$mTypeDataItemIds array in the function initDataTpes in SMWDataValueFactory. Is there anyway i can verify if i have registered this datavalue properly? for example adding a breakpoint to a method that SMW would call in the datavalue? > > > Now for the first issue (which may not be your problem yet, but will > probably come up later). Container-type data values need to be queried > differently. Every datavalue has its own code for deciding what to query > when a user enters some string in #ask. This is implemented in the > SMWDataValue's method getQueryDescription(). The default handling is to > interpret initial comparators (<, >, ...) first, and then parse the rest > like a user input. The result is used for querying. This does not work for > containers, since, IIRC, SMW does not currently support the use of > container DIs in queries (i.e., in the PHP level description of queries). I > might be wrong here. > So far i have just added a stub method getQueryDescription in the datavalue class, however it's not being called (yet), however i'm guessing that is because of my issue above. > > The pedestrian solution for now is to implement something like in > SMW_DV_Record.php. The code there has two modes during parsing: either > construct a DIContainer or construct a query description. This is actually > needed there because Records also support comparators (<, >, ...) for > *each* of their component values in queries; so the parsing is sightly > different (it must allow comparators in front of each value, not just at > the very beginning). If this is not desired, then one could also have the > same parsing and just turn the container into a query (this might already > have code somewhere; not sure). The only thing to do differently would be > to use an anonymous subject for the Container in this case -- otherwise the > query will search for exactly the subject name that you have given, which > is not correct. > I will have to get back to you on this one when i get that far. > > This thread is getting into the most comprehensive documentation of SMW > internals ever written ;-) > > Markus > Cheers. Kim -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
Ok, added the changes in a new branch polygons: these are the new urls: https://gerrit.wikimedia.org/r/9360 (semanticmaps) https://gerrit.wikimedia.org/r/9361 (smw) -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
I tried creating a new branch, however not so successfull. Errors running git rebase -i remotes/gerrit/geopol fatal: Needed a single revision invalid upstream remotes/gerrit/geopol What i ended up with was adding a topic instead as it from the documentation seemed that would be created in its own branch (if it did or not, im not sure.). so anyways, here are my changes thus far. both in SemanticMediaWiki and SemanticMaps: https://gerrit.wikimedia.org/r/#/c/9218/ https://gerrit.wikimedia.org/r/#/c/9221/ please take a look and see if you can't spot whats wrong, also let me know if something is heading in the wrong direction. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
Hey Kim, Great to see you make progress! :) > would it be ok if i added the code ive written so far to gerrit? so you guys can follow my progress line by line? that way you could probably help me better if other issues arise. Yes, it'd be fantastic if you could push your code to Gerrit - I'd be happy to do some review. However, if the code is unstable or unfinished, as is the case here, it cannot be pushed to master. So you should create your own feature branch and then push to that using "git review nameofyourfeaturebranch". I have not tried this myself, but according to people on wikitech this should work :) >> SMWDataItem::TYPE_GEO=> 'smw_coords', // currently created only if Semantic Maps are installed > You could simply require SM as a prerequisite for your code (temporarily, until SMW does the right thing by default), or you could create the same table as SM. I think this documentation is out of date. The table is registered in the store like all others, so should always be created, even if SM is not installed. So SM should not be a prerequisite for handling coordinate data, it only holds DataValues, ValueDescriptions and ResultPrinters. > This thread is getting into the most comprehensive documentation of SMW internals ever written ;-) Yeah :D Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
On 24/05/12 09:41, Kim Eik wrote:
> So I've managed to get past my issues, even without help! ;)
Great :-)
Re your missing table issue: future versions of SMW will always have
coordinate support enabled. Migration code (if needed) will be based on
the SM table. So a good approach might be to store your data in this
table rather than creating a new one (I am not even sure how SMW would
behave when registering two tables that can hold coordinates). You could
simply require SM as a prerequisite for your code (temporarily, until
SMW does the right thing by default), or you could create the same table
as SM.
> Anyways,
> you knock one down, and another one pops up, isn't that how it usually
> is? So what i have done so far is that i put all the coordinates in a
> Container object instead. that solved the one to many mapping,
Yes, that is the right way to go, I think.
> however
> now it seems that i can't retrieve the data when i try to query it
> through the #ask: parser function.
>
> I have a page where i have defined the property
> [[Polygon::,]], the page belongs to a category Polygon.
>
> on the category page i have the following contents:
>
> {{#ask: [[Category:Polygon]]
> | ?Coordinates
> | ?Polygon
> | template=Oilfield-popup
> | format=table
> }}
>
> which draws a table on the cateogory page, however, the column with the
> polygon data is empty.
>
> When digging in the sourcecode i can see that there is a call made to
> SMWResultArray::getNextDataValue(), this however seems to always return
> false instead of the expected container value when it comes to the
> polygon property. Can someone please explain why that is?
There are two things here: using container-type values in query
conditions (querying by their values) and using container-type values in
printouts (displaying their values). The second is not completely clear
to me right now. The DV for Record is getting a Container on
initialisation, so SMW seems to reproduce the complete container in this
case (this might happen in SMW_SqlStubSemanticData.php). I don't know
right now why this would work there and not for your case. Did you
register your new datatype (DV class) to use containers? This is done in
the hook smwInitDatatypes; see SMW_DatavalueFactory for docuemntation
(it's similar to the property registration, using registerDatatype and
registerDatatypeAlias). Without this, SMW won't know that you need a
container and it will just give you the subobject (wikipage DI) when
retireving data from the store; this would then be rejected by your DV,
and no values would be created.
Now for the first issue (which may not be your problem yet, but will
probably come up later). Container-type data values need to be queried
differently. Every datavalue has its own code for deciding what to query
when a user enters some string in #ask. This is implemented in the
SMWDataValue's method getQueryDescription(). The default handling is to
interpret initial comparators (<, >, ...) first, and then parse the rest
like a user input. The result is used for querying. This does not work
for containers, since, IIRC, SMW does not currently support the use of
container DIs in queries (i.e., in the PHP level description of
queries). I might be wrong here.
The pedestrian solution for now is to implement something like in
SMW_DV_Record.php. The code there has two modes during parsing: either
construct a DIContainer or construct a query description. This is
actually needed there because Records also support comparators (<, >,
...) for *each* of their component values in queries; so the parsing is
sightly different (it must allow comparators in front of each value, not
just at the very beginning). If this is not desired, then one could also
have the same parsing and just turn the container into a query (this
might already have code somewhere; not sure). The only thing to do
differently would be to use an anonymous subject for the Container in
this case -- otherwise the query will search for exactly the subject
name that you have given, which is not correct.
This thread is getting into the most comprehensive documentation of SMW
internals ever written ;-)
Markus
> On Wed, May 23, 2012 at 2:23 PM, Kim Eik > wrote:
>
> Ok, so i figured out how to create the necessary table.. however i'm
> unsure of how i should set up a one to many mapping towards the
> sm_coords table..
>
> as of now i have defined the following table:
>
> self::$prop_tables['smw_polygon'] = new SMWSQLStore2Table(
> 'sm_polygon',
> array( 'coordinate_id' => 'p'),
> array( 'coordinate_id' )
> );
>
> where coordinate_id is supposed to refer to sm_coords table entries.
> however, it seems that no other table here has this type of
> structure/mapping. So i'm guessing this is the wrong way to go? I
> could always just have all the data in a clob field instead, but
> this i'm guess
Re: [SMW-devel] Semantic mediawiki and map data
So I've managed to get past my issues, even without help! ;) Anyways, you
knock one down, and another one pops up, isn't that how it usually is? So
what i have done so far is that i put all the coordinates in a Container
object instead. that solved the one to many mapping, however now it seems
that i can't retrieve the data when i try to query it through the #ask:
parser function.
I have a page where i have defined the property
[[Polygon::,]], the page belongs to a category Polygon.
on the category page i have the following contents:
{{#ask: [[Category:Polygon]]
| ?Coordinates
| ?Polygon
| template=Oilfield-popup
| format=table
}}
which draws a table on the cateogory page, however, the column with the
polygon data is empty.
When digging in the sourcecode i can see that there is a call made to
SMWResultArray::getNextDataValue(), this however seems to always return
false instead of the expected container value when it comes to the polygon
property. Can someone please explain why that is?
Kim
On Wed, May 23, 2012 at 2:23 PM, Kim Eik wrote:
> Ok, so i figured out how to create the necessary table.. however i'm
> unsure of how i should set up a one to many mapping towards the sm_coords
> table..
>
> as of now i have defined the following table:
>
> self::$prop_tables['smw_polygon'] = new SMWSQLStore2Table(
> 'sm_polygon',
> array( 'coordinate_id' => 'p'),
> array( 'coordinate_id' )
> );
>
> where coordinate_id is supposed to refer to sm_coords table entries.
> however, it seems that no other table here has this type of
> structure/mapping. So i'm guessing this is the wrong way to go? I could
> always just have all the data in a clob field instead, but this i'm
> guessing isn't very optimal when taking semantic search into account.
>
> So how should i create a one to many mapping on the database level as well
> as on the logic level? or should polygons be stored in some other way?
>
> Cheers.
>
> On Wed, May 23, 2012 at 12:49 PM, Kim Eik wrote:
>
>> Ok, so i have run into an issue.
>>
>> A database query syntax error has occurred. This may indicate a bug in
>> the software. The last attempted database query was:
>>
>> (SQL query hidden)
>>
>> from within function "SMW::getPropertySubjects". Database returned error
>> "1103: Incorrect table name '' (localhost)".
>>
>> What i have done so far is defined a new property Geographic polygon. and
>> in my test wiki i tried setting the property [[Has type::Geographical
>> polygon]]
>>
>> By doing that i'm now getting this database issue. From what i can gather
>> i'm missing a database table to store my structure.
>> In the existing code i can see tht TYPE_GEO is defined as smw_coords
>> which im guessing is the table name that stores Geographic coordinates.
>>
>> SMWDataItem::TYPE_GEO=> 'smw_coords', // currently created only
>> if Semantic Maps are installed
>>
>> Now, how should i proceed to add a table for a polygon?
>> a polygon is just a collection of coords, so i guess i could use
>> smw_coords as my main table for storing the coordinates of polygons, and
>> then another table that refers to smw_coords with a one to many mapping.
>>
>> And where in the semantic maps extension can i actually find som sql code
>> that creates the given table?
>>
>> Oh, and btw, i could use some code review as im piecing this together.
>> would it be ok if i added the code ive written so far to gerrit? so you
>> guys can follow my progress line by line? that way you could probably help
>> me better if other issues arise. So far i have written code for
>> SemanticMediaWiki and SemanticMaps.
>>
>> Cheers
>>
>>
>> On Wed, May 23, 2012 at 11:11 AM, Markus Krötzsch <
>> mar...@semantic-mediawiki.org> wrote:
>>
>>> On 23/05/12 09:02, Kim Eik wrote:
>>>
>From your descritpion, a container data item looks like the way to go
for storing my structures. Do you have a practical example on how
container data item is used?
From the code documentation i can see that:
* Being a mere placeholder/template for other data, an SMWDIContainer
is not
* immutable as the other basic data items. New property-value pairs
can always
* be added to the internal SMWContainerSemanticData.
so is it correct to assume that in a practical example, in order to
create a container data item, you actually create a template which holds
all your other properties (like a set of [[Polygon::...]] properties)?
>>>
>>> The word "template" is not well chosen in this documentation. It has
>>> nothing to do with a MW template. When using containers to store data, they
>>> are simply like an "internal wiki page" together with property-value
>>> assignments. Instead of using containers, one could also create a new wiki
>>> page, assign all the data to that wiki page, and then use the wiki page
>>> instead of the Container DI as a property value. But this would not fit
>>> into one property assignment and would require multip
Re: [SMW-devel] Semantic mediawiki and map data
Ok, so i figured out how to create the necessary table.. however i'm unsure
of how i should set up a one to many mapping towards the sm_coords table..
as of now i have defined the following table:
self::$prop_tables['smw_polygon'] = new SMWSQLStore2Table(
'sm_polygon',
array( 'coordinate_id' => 'p'),
array( 'coordinate_id' )
);
where coordinate_id is supposed to refer to sm_coords table entries.
however, it seems that no other table here has this type of
structure/mapping. So i'm guessing this is the wrong way to go? I could
always just have all the data in a clob field instead, but this i'm
guessing isn't very optimal when taking semantic search into account.
So how should i create a one to many mapping on the database level as well
as on the logic level? or should polygons be stored in some other way?
Cheers.
On Wed, May 23, 2012 at 12:49 PM, Kim Eik wrote:
> Ok, so i have run into an issue.
>
> A database query syntax error has occurred. This may indicate a bug in the
> software. The last attempted database query was:
>
> (SQL query hidden)
>
> from within function "SMW::getPropertySubjects". Database returned error
> "1103:
> Incorrect table name '' (localhost)".
>
> What i have done so far is defined a new property Geographic polygon. and
> in my test wiki i tried setting the property [[Has type::Geographical
> polygon]]
>
> By doing that i'm now getting this database issue. From what i can gather
> i'm missing a database table to store my structure.
> In the existing code i can see tht TYPE_GEO is defined as smw_coords which
> im guessing is the table name that stores Geographic coordinates.
>
> SMWDataItem::TYPE_GEO=> 'smw_coords', // currently created only if
> Semantic Maps are installed
>
> Now, how should i proceed to add a table for a polygon?
> a polygon is just a collection of coords, so i guess i could use
> smw_coords as my main table for storing the coordinates of polygons, and
> then another table that refers to smw_coords with a one to many mapping.
>
> And where in the semantic maps extension can i actually find som sql code
> that creates the given table?
>
> Oh, and btw, i could use some code review as im piecing this together.
> would it be ok if i added the code ive written so far to gerrit? so you
> guys can follow my progress line by line? that way you could probably help
> me better if other issues arise. So far i have written code for
> SemanticMediaWiki and SemanticMaps.
>
> Cheers
>
>
> On Wed, May 23, 2012 at 11:11 AM, Markus Krötzsch <
> mar...@semantic-mediawiki.org> wrote:
>
>> On 23/05/12 09:02, Kim Eik wrote:
>>
>>> >From your descritpion, a container data item looks like the way to go
>>> for storing my structures. Do you have a practical example on how
>>> container data item is used?
>>>
>>> From the code documentation i can see that:
>>>
>>> * Being a mere placeholder/template for other data, an SMWDIContainer
>>> is not
>>> * immutable as the other basic data items. New property-value pairs
>>> can always
>>> * be added to the internal SMWContainerSemanticData.
>>>
>>> so is it correct to assume that in a practical example, in order to
>>> create a container data item, you actually create a template which holds
>>> all your other properties (like a set of [[Polygon::...]] properties)?
>>>
>>
>> The word "template" is not well chosen in this documentation. It has
>> nothing to do with a MW template. When using containers to store data, they
>> are simply like an "internal wiki page" together with property-value
>> assignments. Instead of using containers, one could also create a new wiki
>> page, assign all the data to that wiki page, and then use the wiki page
>> instead of the Container DI as a property value. But this would not fit
>> into one property assignment and would require multiple operations instead.
>> Containers are mainly a convenience structure so that data for multiple
>> (sub)wikipages can be processed in one go.
>>
>> The word "template" comes in when you use a container value as a search
>> pattern (in queries or in special pages). Then a container can be used
>> without a subject ("anonymous" subject): in this case the subject's name
>> does not matter in the search and only the property values of the container
>> are used to find matches. This is what is meant by "template" in this
>> sentence.
>>
>> You can see a short code example on how to create arbitrary container DIs
>> in the file SMW_Subobject.php (the implementation of the subobject parser
>> function). This code uses a user-provided name for the subobject. There is
>> also code in SMW_DV_Record.php that generates an internal name on the fly
>> by hashing the data (around line 42). Generated names should start with
>> "_". This allows SMW to decide if a subobject name should be displayed to
>> users or not.
>>
>>
>>
>>> As i said, i'm quite new to this semantic way of structuring data, so
>>> please forgive me for not having a better understanding of how all this
>
Re: [SMW-devel] Semantic mediawiki and map data
Ok, so i have run into an issue.
A database query syntax error has occurred. This may indicate a bug in the
software. The last attempted database query was:
(SQL query hidden)
from within function "SMW::getPropertySubjects". Database returned error "1103:
Incorrect table name '' (localhost)".
What i have done so far is defined a new property Geographic polygon. and
in my test wiki i tried setting the property [[Has type::Geographical
polygon]]
By doing that i'm now getting this database issue. From what i can gather
i'm missing a database table to store my structure.
In the existing code i can see tht TYPE_GEO is defined as smw_coords which
im guessing is the table name that stores Geographic coordinates.
SMWDataItem::TYPE_GEO=> 'smw_coords', // currently created only if
Semantic Maps are installed
Now, how should i proceed to add a table for a polygon?
a polygon is just a collection of coords, so i guess i could use smw_coords
as my main table for storing the coordinates of polygons, and then another
table that refers to smw_coords with a one to many mapping.
And where in the semantic maps extension can i actually find som sql code
that creates the given table?
Oh, and btw, i could use some code review as im piecing this together.
would it be ok if i added the code ive written so far to gerrit? so you
guys can follow my progress line by line? that way you could probably help
me better if other issues arise. So far i have written code for
SemanticMediaWiki and SemanticMaps.
Cheers
On Wed, May 23, 2012 at 11:11 AM, Markus Krötzsch <
mar...@semantic-mediawiki.org> wrote:
> On 23/05/12 09:02, Kim Eik wrote:
>
>> >From your descritpion, a container data item looks like the way to go
>> for storing my structures. Do you have a practical example on how
>> container data item is used?
>>
>> From the code documentation i can see that:
>>
>> * Being a mere placeholder/template for other data, an SMWDIContainer
>> is not
>> * immutable as the other basic data items. New property-value pairs
>> can always
>> * be added to the internal SMWContainerSemanticData.
>>
>> so is it correct to assume that in a practical example, in order to
>> create a container data item, you actually create a template which holds
>> all your other properties (like a set of [[Polygon::...]] properties)?
>>
>
> The word "template" is not well chosen in this documentation. It has
> nothing to do with a MW template. When using containers to store data, they
> are simply like an "internal wiki page" together with property-value
> assignments. Instead of using containers, one could also create a new wiki
> page, assign all the data to that wiki page, and then use the wiki page
> instead of the Container DI as a property value. But this would not fit
> into one property assignment and would require multiple operations instead.
> Containers are mainly a convenience structure so that data for multiple
> (sub)wikipages can be processed in one go.
>
> The word "template" comes in when you use a container value as a search
> pattern (in queries or in special pages). Then a container can be used
> without a subject ("anonymous" subject): in this case the subject's name
> does not matter in the search and only the property values of the container
> are used to find matches. This is what is meant by "template" in this
> sentence.
>
> You can see a short code example on how to create arbitrary container DIs
> in the file SMW_Subobject.php (the implementation of the subobject parser
> function). This code uses a user-provided name for the subobject. There is
> also code in SMW_DV_Record.php that generates an internal name on the fly
> by hashing the data (around line 42). Generated names should start with
> "_". This allows SMW to decide if a subobject name should be displayed to
> users or not.
>
>
>
>> As i said, i'm quite new to this semantic way of structuring data, so
>> please forgive me for not having a better understanding of how all this
>> is connected.
>>
>> And another thing, in which end would you say i should start trying to
>> implement my data structure, should i start with the individual
>> dataitems/datavalues and then look at creating a container? do i even
>> need to create/subclass my own container? or can i use the one already
>> defined?
>>
>
> You can use the one that is already defined. In order to store data in a
> container, you need properties. You need to register these properties so
> that they are available (and have the correct type) without the user having
> to create pages for them. Every property has an internal string ID that you
> use in the code to address it. It can also have a user-label that users can
> write to address it (if no label is given, then users will never see it and
> cannot search for it).
>
> Property registration is done in a hook, for example:
>
> $wgHooks['smwInitProperties'][**] = 'myinitProperties';
>
> function myinitProperties() {
> // Register a user-visible property w
Re: [SMW-devel] Semantic mediawiki and map data
On 23/05/12 09:02, Kim Eik wrote:
> >From your descritpion, a container data item looks like the way to go
> for storing my structures. Do you have a practical example on how
> container data item is used?
>
> From the code documentation i can see that:
>
> * Being a mere placeholder/template for other data, an SMWDIContainer
> is not
> * immutable as the other basic data items. New property-value pairs
> can always
> * be added to the internal SMWContainerSemanticData.
>
> so is it correct to assume that in a practical example, in order to
> create a container data item, you actually create a template which holds
> all your other properties (like a set of [[Polygon::...]] properties)?
The word "template" is not well chosen in this documentation. It has
nothing to do with a MW template. When using containers to store data,
they are simply like an "internal wiki page" together with
property-value assignments. Instead of using containers, one could also
create a new wiki page, assign all the data to that wiki page, and then
use the wiki page instead of the Container DI as a property value. But
this would not fit into one property assignment and would require
multiple operations instead. Containers are mainly a convenience
structure so that data for multiple (sub)wikipages can be processed in
one go.
The word "template" comes in when you use a container value as a search
pattern (in queries or in special pages). Then a container can be used
without a subject ("anonymous" subject): in this case the subject's name
does not matter in the search and only the property values of the
container are used to find matches. This is what is meant by "template"
in this sentence.
You can see a short code example on how to create arbitrary container
DIs in the file SMW_Subobject.php (the implementation of the subobject
parser function). This code uses a user-provided name for the subobject.
There is also code in SMW_DV_Record.php that generates an internal name
on the fly by hashing the data (around line 42). Generated names should
start with "_". This allows SMW to decide if a subobject name should be
displayed to users or not.
>
> As i said, i'm quite new to this semantic way of structuring data, so
> please forgive me for not having a better understanding of how all this
> is connected.
>
> And another thing, in which end would you say i should start trying to
> implement my data structure, should i start with the individual
> dataitems/datavalues and then look at creating a container? do i even
> need to create/subclass my own container? or can i use the one already
> defined?
You can use the one that is already defined. In order to store data in a
container, you need properties. You need to register these properties so
that they are available (and have the correct type) without the user
having to create pages for them. Every property has an internal string
ID that you use in the code to address it. It can also have a user-label
that users can write to address it (if no label is given, then users
will never see it and cannot search for it).
Property registration is done in a hook, for example:
$wgHooks['smwInitProperties'][] = 'myinitProperties';
function myinitProperties() {
// Register a user-visible property with ID '___myp':
SMWDIProperty::registerProperty('___myp', '_num', 'Myproperty', true);
// Register an alternative user label for this property:
SMWDIProperty::registerPropertyAlias('___myp', 'My property');
return true;
}
See the documentation for registerProperty() for details. Always call
your own property IDs with three initial underscores to avoid name clashes.
After you registeed properties in this way, you can use them for storing
and retrieving values in the container (if they have a user label, you
can also use them in the wiki; a good way to try if it worked). In PHP,
you can create a property object like this:
$myProperty = new SMWDIProperty('___myp');
Do not use your label but the property ID here. To create a property
from its label, you could use the method
SMWDIProperty::newFromUserLabel(), but this should not be needed. If the
property is registered, SMW will use the right datatype for storing and
retrieving it. If the datatype is changed later on, SMW will not find
old stored values any more (as expected; if you store values as a
string, you don't get them when looking for numbers later on). However,
the problem repairs itself over time (old values get deleted when pages
are stored again).
Moreover, you can add any value for any property in a Container -- it is
not checked if the types are the same (e.g., you can assign a DIString
object to a property that is declared to hold a number). If types are
not the same, the data might simply not be stored, or might be stored in
the wrong place and not be found again. Again, this will repair itself
when correcting the code and not cause permanent problems.
One way to inspect the
Re: [SMW-devel] Semantic mediawiki and map data
>From your descritpion, a container data item looks like the way to go for storing my structures. Do you have a practical example on how container data item is used? >From the code documentation i can see that: * Being a mere placeholder/template for other data, an SMWDIContainer is not * immutable as the other basic data items. New property-value pairs can always * be added to the internal SMWContainerSemanticData. so is it correct to assume that in a practical example, in order to create a container data item, you actually create a template which holds all your other properties (like a set of [[Polygon::...]] properties)? As i said, i'm quite new to this semantic way of structuring data, so please forgive me for not having a better understanding of how all this is connected. And another thing, in which end would you say i should start trying to implement my data structure, should i start with the individual dataitems/datavalues and then look at creating a container? do i even need to create/subclass my own container? or can i use the one already defined? Best regards from mr. slightly confused On Tue, May 22, 2012 at 9:48 PM, Markus Krötzsch < mar...@semantic-mediawiki.org> wrote: > Hi Kim, > > that is a very exciting project. Here are some general hints/comments: > > * Jeroen is right, but I would suggest a slightly different approach :-) > > * To get started, you can store your data in a Container data item > (similar to subobjects and records). In essence, this means that each value > can be an arbitrary collection of property-value pairs. This gives you most > flexibility and might even be needed for things like polygons where you > have no limit on the amount of data. > > * This does not affect your surface syntax. You can create a datavalue > (DV) implementation for your datatypes, and the parsing and general I/O > will be the same in any case. So it will allow you to do many things. > > * Once this works and you have a clearer idea what the data looks like, it > might be useful to have special DIs for it. The advantage of that is that > the database can handle them more efficiently than general purpose > container objects. The disadvantage is that the database would need to > handle them in special ways instead of treating them as general purpose > container objects. So there is some code needed to get update, retrieval, > and query answering implemented. > > * The special DIs can still support the interface of Container, they can > even be a special form of containers, and thus create subobjects to which > further properties could be attached. This does not prevent the database > from having a special handling for these containers if it is known that > their structure is of a special form that could be stored in a more > efficient way. So it is not necessary that your code avoids the use of > Container-specific code just because of the possible later change to a > custom DI. In fact, you will certainly need to use the interface of > Container to get at the data. > > * Nischay is currently working on a general overhaul of the SQL database > backend as part of his Google Summer of Code project. This might simplify > such extensions in the future by making the code better structured and more > accessible. So there are good reasons to wait with DI extensions now. We > will keep this use case in mind -- maybe the code that is specific to each > DI can be cleanly separated from the rest. > > Cheers, > > Markus > > > > On 22/05/12 16:08, Jeroen De Dauw wrote: > >> Hey, >> >> > a map can have several polygons >> >> Sure. Which polygons a map shows is determined by the query it >> visualizes the results for. When creating the definitions of the >> polygons you should not have to care to much of which polygon will end >> up on what map. You just create a data structure that makes sense and >> ensure that you can actually do the kind of queries you want to do. >> >> > Would this structure be a fair assumption? >> > >> [[Polygons::[[Polygon::<**coords>,,...]]** >> ,[[Polygon::,,**...]],[[Polygon::<** >> coords>,,...]]**]] >> >> No. Imagine you have a pile of articles of cities, then you could have a >> property such as "Has location" (which is of type polygon) which would >> look like: >> >> [[Has location::,,<**coords>...]] >> >> If you have an article describing an object that has multiple polygons, >> for example a group of islands where each island needs a polygon, then >> you simply have multiple such properties on the page. >> >> > So say i have a [[Polygon::,,<**coords>]] property >> defined exactly how can i point additional metadata to this object? >> >> In order to be able to point to an object, it needs to be of type page. >> So you cannot out of the box specify a property such as "Has stroke >> color". One option is to specify the polygons as part of subobjects [0], >> which can then also hold any extra extra info on the polygon. >> >> In a way it would be nicer to not have it as
Re: [SMW-devel] Semantic mediawiki and map data
Hi Kim, that is a very exciting project. Here are some general hints/comments: * Jeroen is right, but I would suggest a slightly different approach :-) * To get started, you can store your data in a Container data item (similar to subobjects and records). In essence, this means that each value can be an arbitrary collection of property-value pairs. This gives you most flexibility and might even be needed for things like polygons where you have no limit on the amount of data. * This does not affect your surface syntax. You can create a datavalue (DV) implementation for your datatypes, and the parsing and general I/O will be the same in any case. So it will allow you to do many things. * Once this works and you have a clearer idea what the data looks like, it might be useful to have special DIs for it. The advantage of that is that the database can handle them more efficiently than general purpose container objects. The disadvantage is that the database would need to handle them in special ways instead of treating them as general purpose container objects. So there is some code needed to get update, retrieval, and query answering implemented. * The special DIs can still support the interface of Container, they can even be a special form of containers, and thus create subobjects to which further properties could be attached. This does not prevent the database from having a special handling for these containers if it is known that their structure is of a special form that could be stored in a more efficient way. So it is not necessary that your code avoids the use of Container-specific code just because of the possible later change to a custom DI. In fact, you will certainly need to use the interface of Container to get at the data. * Nischay is currently working on a general overhaul of the SQL database backend as part of his Google Summer of Code project. This might simplify such extensions in the future by making the code better structured and more accessible. So there are good reasons to wait with DI extensions now. We will keep this use case in mind -- maybe the code that is specific to each DI can be cleanly separated from the rest. Cheers, Markus On 22/05/12 16:08, Jeroen De Dauw wrote: > Hey, > > > a map can have several polygons > > Sure. Which polygons a map shows is determined by the query it > visualizes the results for. When creating the definitions of the > polygons you should not have to care to much of which polygon will end > up on what map. You just create a data structure that makes sense and > ensure that you can actually do the kind of queries you want to do. > > > Would this structure be a fair assumption? > > > [[Polygons::[[Polygon::,,...]],[[Polygon::,,...]],[[Polygon::,,... > > No. Imagine you have a pile of articles of cities, then you could have a > property such as "Has location" (which is of type polygon) which would > look like: > > [[Has location::,,...]] > > If you have an article describing an object that has multiple polygons, > for example a group of islands where each island needs a polygon, then > you simply have multiple such properties on the page. > > > So say i have a [[Polygon::,,]] property > defined exactly how can i point additional metadata to this object? > > In order to be able to point to an object, it needs to be of type page. > So you cannot out of the box specify a property such as "Has stroke > color". One option is to specify the polygons as part of subobjects [0], > which can then also hold any extra extra info on the polygon. > > In a way it would be nicer to not have it as part of the subobject, but > it actually behave like one, in which case you could have a list of "Has > point" properties. This way you could do stuff like querying all the > points in a polygon that have a latitude more then x. This requires some > more thought I think - anyone more thoughts on this? > > [0] http://semantic-mediawiki.org/wiki/Subobject > > Cheers > > -- > Jeroen De Dauw > http://www.bn2vs.com > Don't panic. Don't be evil. > -- > > > -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > > > ___ > Semediawiki-devel mailing list > Semediawiki-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/semediawiki-devel -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and
Re: [SMW-devel] Semantic mediawiki and map data
Hey, > a map can have several polygons Sure. Which polygons a map shows is determined by the query it visualizes the results for. When creating the definitions of the polygons you should not have to care to much of which polygon will end up on what map. You just create a data structure that makes sense and ensure that you can actually do the kind of queries you want to do. > Would this structure be a fair assumption? > [[Polygons::[[Polygon::,,...]],[[Polygon::,,...]],[[Polygon::,,... No. Imagine you have a pile of articles of cities, then you could have a property such as "Has location" (which is of type polygon) which would look like: [[Has location::,,...]] If you have an article describing an object that has multiple polygons, for example a group of islands where each island needs a polygon, then you simply have multiple such properties on the page. > So say i have a [[Polygon::,,]] property defined exactly how can i point additional metadata to this object? In order to be able to point to an object, it needs to be of type page. So you cannot out of the box specify a property such as "Has stroke color". One option is to specify the polygons as part of subobjects [0], which can then also hold any extra extra info on the polygon. In a way it would be nicer to not have it as part of the subobject, but it actually behave like one, in which case you could have a list of "Has point" properties. This way you could do stuff like querying all the points in a polygon that have a latitude more then x. This requires some more thought I think - anyone more thoughts on this? [0] http://semantic-mediawiki.org/wiki/Subobject Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
Thank you for your introduction Jeroen, i shall gaze upon the smw sourcecode with new light tomorrow and see if i can't implement something along the lines of your suggestion. A couple of questions though. As for your insight on the metadata issue, you state that it would be better to have metadata pointing to it. So say i have a [[Polygon::,,]] property defined exactly how can i point additional metadata to this object? properties like stroke color, stroke weight, stroke opacity, fill color, fill opacity but also textual content (title/text). And How does smw handle one to many relationships? take for example a the Maps polygons parameter, a map can have several polygons, one polygon consists of several coordinates. Would smw be able to handle a collection of polygons which is a collection of coordinates? How would this be defined in wiki code? Would this structure be a fair assumption? [[Polygons::[[Polygon::,,...]],[[Polygon::,,...]],[[Polygon::,,... Best regards -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
Hey, This is indeed a very exciting feature and I can't wait to see it implemented :) > Should i define each one of these as a SMWDataItem::TYPE_xxx? Or should these things be defined through things like a combination of record/blobs and arraymap? Having dedicated data types seems like the way to go - if you simply use compounds or subobjects, then it'll not be possible for the SMW stores to nicely optimize for this kind of data (ie apply the appropriate geospatial indexes). Since SMW 1.6 you can no longer extend the datatypes from outside (for good reasons), so you'd be creating your own SMWDataItem deriving classes in SMW. Once these are done, you need a way to actually represent this data to the user, which is done by creating so called DataValues, which derive from SMWDataValue and typically hold a SMWDataItem (more info on this here [0]). These DataValues should go into Semantic Maps (where the current GeoCoord DataValue resides as well). One final element you need is a way to actually set these kind of values. This is also done in the DataValue, which specifies how values of it's type are parsed. Depending on just what kind of data these objects will hold, it might not might not be useful to create one or more dedicated parser functions instead of just extending the extended link syntax. If you just have points, this is not needed, but if you have points which can have all kinds of additional info associated (including semi-free-text descriptions), then a dedicated parser function might be better. One more though I have on this is that if you have additional data that is associated with the geospatial shape as a whole, it'd be better to have it as other properties pointing to the object. This way you allow people to have whatever type of extra info they want. You of course will need to know which data to use for what when the actual map is being build, which can be done by having additional parameters in the #ask function that specify "the value of property Has description should be used as description", ie descrtiptionproperty=Has description. [0] There is some info about this here: http://semantic-mediawiki.org/wiki/Architecture_guide Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] Semantic mediawiki and map data
I can't really help you, but I'm looking forward to seeing the result of your work! This is something I have wanted for a long time, so just tell me when you need help beta testing it! /Leo Wallentin ___Leonard wallentinleo_wallen...@hotmail.com@leo_wallentin+46 (0) 735 - 933 543 http://svt.se/nyhetslabbethttp://säsongsmat.nuhttp://nairobikoll.se Date: Tue, 22 May 2012 09:57:08 +0200 From: k...@heldig.org To: semediawiki-devel@lists.sourceforge.net Subject: [SMW-devel] Semantic mediawiki and map data I'm rather new to semantic mediawiki and seeking some assistance in terms on how i should proceed. Today, semantic mediawiki (through the Semantic Maps extension) supports defining geographic coordinates. a single point on a map. However, i intend to expand on this feature and allow to define areas on a map through the use of polygons, circles, rectangles and lines. So let's look at the individual datatypes:Polygons and lines are just a non-finite ordered list of coordinatesCircles is defined through a single point/coordinate and a radius. Rectangles is defined through two points on a map So, somehow i need to define these datatypes, what would be the best way to do this? any ideas? Should i define each one of these as a SMWDataItem::TYPE_xxx? Or should these things be defined through things like a combination of record/blobs and arraymap? Any suggestions on this is greatly appreciated as i'm not sure what path i should take. Best regards Kim -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
