Re: [SMW-devel] [News] SVN update warning
Hi all, Forgive me if I misunderstand or am off topic The idea of internal objects is very interesting and I appreciate the description of SIO versus SMW native objects. What I am wondering is if the idea of internal objects could be extended to complex objects like tables. One could imagine a whole set of capabilities oriented around tables of data entered by users and/or extracted from collections of pages. In the latter case, you would not want to be constrained to a 1 to 1 mapping between page and object. And in either case, you would not want the object stored as a 256 character string. I'm curious as to whether you have considered this type of capability and how will it does (or doesn't) fit within your design or future plans. Karen On 1/6/10 12:17 PM, "Markus Krötzsch" wrote: On Mittwoch, 6. Januar 2010, Patrick Nagel wrote: > Hi Markus, > > On 2010-01-06 09:27 UTC Markus Krötzsch wrote: > > The change will also introduce some fixed limit on how many values a > > multi- valued property can at most have. It is currently set to 5. > > If you have any such properties with more than 5 values then please > > let me know. > > Is this really necessary? I once had an application that would have used > around 30 values in one multi-valued property (project details to be > stored on an invoice page - which can cover multiple projects) - I > didn't go for multi-valued properties though, since it's impossible to > get the data out again in a useful way (which has been brought up a lot > of times on these lists, by me and many others). I'm using Semantic > Internal Objects now, but as Yaron pointed out, there are issues with > it, that would require changes in SMW. > > I really hope that your recent code changes will finally make it > possible to have "bundles of values that belong together" stored in a > property (or an internal object, or whatever other thing), and that it's > possible to get them out again in a useful way - not introduce new > limitations to that feature, that is really necessary as soon as you do > something non-trivial with SMW. > > If there must be a fixed limit, couldn't it be something like 2^14 - > like Excel's maximum amount of columns (iirc)? In fact, a main purpose of change in SMW is to make way for native "internal objects" which are rather similar to what Semantic Internal Objects does. The difference that remains is that SIO makes an internal object that has a property *to* the page that it was created on, while internal objects in SMW must have a property pointing to the object *from* the page that it was created on (SMW does not currently have a way to properly support the inverted property style of SIO in a way that avoids the bugs that Yaron mentioned earlier). The new code is able to handle arbitrary property-value bundles natively in SMW. A surface syntax for this remains to be implemented, but all essential related to storage and querying of such data is ready. It might happen that multi-valued properties are eventually going to be replaced by this new system altogether, or that they co-exist with various other forms of compound data values. To make multi-valued properties fit into this scheme, however, we need to consider them as internal objects that have values assigned to internal auxiliary properties "_1", "_2", "_3", ... So multi-valued properties are really encoded as internal objects already, with the only peculiarity that the properties are assigned in a hard-coded way and are not entered by the user. We currently have all the special properties "_1", "_2", ... pre-defined in SMW, so that retrieving them even requires one DB lookup less than for the old nary code. If we could agree on some reasonably small upper bound, then I could keep it like this. As I said, it is planed anyway to support a native form of "semantic internal objects" in SMW. I don't think that the input syntax of current multi-valued property data is suitable for anything beyond 10 values. There is also a principal limitation to the number of entries that the type declaration of a multi-valued property can have: the DB column used to store this type string has at most 256 characters. So 51 values are already the theoretical upper limit since even the internal standard types need at least 4 characters and one comma per entry. Markus -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Sem
Re: [SMW-devel] [News] SVN update warning
On Mittwoch, 6. Januar 2010, zehet...@molgen.mpg.de wrote: > Hi, > > I'm not sure what the maximum is right now, between 10 and 20 but the point > is that tomorrow a dataset could come along which has 50 or more parts. So > having any fixed value, especially one which can only be changed by > modifying the code itself, is a serious drawback although I assume you > have a reasons why this limitation is necessary. It would, howerver, be > great to have at least a global variable so one doesn't have to patch the > code after each update. As I just wrote in my other email, the limitation is really part of a change that waives all kinds of existing limitations on the data that SMW can handle in principle. The limitation rather expresses the fact that we should probably limit the resources (both in terms of LOC and in terms of developers' time) that are to be put into multi-valued properties, given that the use cases with large numbers of values would probably better be served by a proper internal object datatype that also remains to be implemented. -- Markus > > Cheers, > Gu > > Quoting Markus Krötzsch : > > On Mittwoch, 6. Januar 2010, zehet...@molgen.mpg.de wrote: > > > Hi, > > > > > > > The change will also introduce some fixed limit on how many values a > > > > multi- valued property can at most have. It is currently set to 5. If > > > > you > > > > > > have any such properties with more than 5 values then please let me > > > > know. > > > > > If I understand correctly you mean the number of value-parts a > > > > multi-valued > > > > > property can have. Most of my properties are multivalued and some > > > > certainly > > > > > have more than just 5 parts. I really hope that 'limit' will be a > > > > variable > > > > > set via LocalSettings (for my use of SMW this change sounds rather > > > scary and makes me really kind of nervous). > > > > So far, the limit is not planned to be a variable, but we can set a > > higher value right away if you need it. How many types do your longest > > n-ary properties have? > > > > Cheers, > > > > Markus > > > > > Cheers, > > > Gu > > > > > > Quoting Markus Krötzsch : > > > > In preparation of SMW 1.5, I am currently doing some large-scale > > > > changes to certain internals of the SMW code. This involves changes > > > > in the way > > > > in > > > > > > which > > > > > > > > multi-valued properties (a.k.a. n-ary properties) are handled. > > > > Eventually, only very little will change from a user's perspective > > > > (only a small change on > > > > the property page of each multi-valued properties will be needed), > > > > but a lot > > > > > > > > will change internally. > > > > > > > > The change will also introduce some fixed limit on how many values a > > > > multi- valued property can at most have. It is currently set to 5. If > > > > you > > > > > > have any such properties with more than 5 values then please let me > > > > know. > > > > > > I therefore suggest to not upgrade wikis that use such properties to > > > > the development version of SMW until the details on how to modify the > > > > pages have > > > > > > > > been published and everything is known to work again. > > > > > > > > I will send a separate email to the developers' list regarding > > > > changes that might affect developers. > > > > > > > > Cheers, > > > > > > > > Markus > -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] [News] SVN update warning
On Mittwoch, 6. Januar 2010, Patrick Nagel wrote: > Hi Markus, > > On 2010-01-06 09:27 UTC Markus Krötzsch wrote: > > The change will also introduce some fixed limit on how many values a > > multi- valued property can at most have. It is currently set to 5. > > If you have any such properties with more than 5 values then please > > let me know. > > Is this really necessary? I once had an application that would have used > around 30 values in one multi-valued property (project details to be > stored on an invoice page - which can cover multiple projects) - I > didn't go for multi-valued properties though, since it's impossible to > get the data out again in a useful way (which has been brought up a lot > of times on these lists, by me and many others). I'm using Semantic > Internal Objects now, but as Yaron pointed out, there are issues with > it, that would require changes in SMW. > > I really hope that your recent code changes will finally make it > possible to have "bundles of values that belong together" stored in a > property (or an internal object, or whatever other thing), and that it's > possible to get them out again in a useful way - not introduce new > limitations to that feature, that is really necessary as soon as you do > something non-trivial with SMW. > > If there must be a fixed limit, couldn't it be something like 2^14 - > like Excel's maximum amount of columns (iirc)? In fact, a main purpose of change in SMW is to make way for native "internal objects" which are rather similar to what Semantic Internal Objects does. The difference that remains is that SIO makes an internal object that has a property *to* the page that it was created on, while internal objects in SMW must have a property pointing to the object *from* the page that it was created on (SMW does not currently have a way to properly support the inverted property style of SIO in a way that avoids the bugs that Yaron mentioned earlier). The new code is able to handle arbitrary property-value bundles natively in SMW. A surface syntax for this remains to be implemented, but all essential related to storage and querying of such data is ready. It might happen that multi-valued properties are eventually going to be replaced by this new system altogether, or that they co-exist with various other forms of compound data values. To make multi-valued properties fit into this scheme, however, we need to consider them as internal objects that have values assigned to internal auxiliary properties "_1", "_2", "_3", ... So multi-valued properties are really encoded as internal objects already, with the only peculiarity that the properties are assigned in a hard-coded way and are not entered by the user. We currently have all the special properties "_1", "_2", ... pre-defined in SMW, so that retrieving them even requires one DB lookup less than for the old nary code. If we could agree on some reasonably small upper bound, then I could keep it like this. As I said, it is planed anyway to support a native form of "semantic internal objects" in SMW. I don't think that the input syntax of current multi-valued property data is suitable for anything beyond 10 values. There is also a principal limitation to the number of entries that the type declaration of a multi-valued property can have: the DB column used to store this type string has at most 256 characters. So 51 values are already the theoretical upper limit since even the internal standard types need at least 4 characters and one comma per entry. Markus -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] [News] SVN update warning
Hi, I'm not sure what the maximum is right now, between 10 and 20 but the point is that tomorrow a dataset could come along which has 50 or more parts. So having any fixed value, especially one which can only be changed by modifying the code itself, is a serious drawback although I assume you have a reasons why this limitation is necessary. It would, howerver, be great to have at least a global variable so one doesn't have to patch the code after each update. Cheers, Gu Quoting Markus Krötzsch : > On Mittwoch, 6. Januar 2010, zehet...@molgen.mpg.de wrote: > > Hi, > > > > > The change will also introduce some fixed limit on how many values a > > > multi- valued property can at most have. It is currently set to 5. If > you > > > have any such properties with more than 5 values then please let me > know. > > > > If I understand correctly you mean the number of value-parts a > multi-valued > > property can have. Most of my properties are multivalued and some > certainly > > have more than just 5 parts. I really hope that 'limit' will be a > variable > > set via LocalSettings (for my use of SMW this change sounds rather scary > > and makes me really kind of nervous). > > So far, the limit is not planned to be a variable, but we can set a higher > value right away if you need it. How many types do your longest n-ary > properties have? > > Cheers, > > Markus > > > > > Cheers, > > Gu > > > > Quoting Markus Krötzsch : > > > In preparation of SMW 1.5, I am currently doing some large-scale changes > > > to certain internals of the SMW code. This involves changes in the way > in > > > which > > > > > > multi-valued properties (a.k.a. n-ary properties) are handled. > > > Eventually, only very little will change from a user's perspective (only > > > a small change on > > > the property page of each multi-valued properties will be needed), but a > > > lot > > > > > > will change internally. > > > > > > The change will also introduce some fixed limit on how many values a > > > multi- valued property can at most have. It is currently set to 5. If > you > > > have any such properties with more than 5 values then please let me > know. > > > > > > I therefore suggest to not upgrade wikis that use such properties to the > > > development version of SMW until the details on how to modify the pages > > > have > > > > > > been published and everything is known to work again. > > > > > > I will send a separate email to the developers' list regarding changes > > > that might affect developers. > > > > > > Cheers, > > > > > > Markus > > > > > -- > Markus Krötzsch > * Personal page: http://korrekt.org > * Semantic MediaWiki: http://semantic-mediawiki.org > * Semantic Web textbook: http://semantic-web-book.org > -- > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] [News] SVN update warning
Hi Markus, On 2010-01-06 09:27 UTC Markus Krötzsch wrote: > The change will also introduce some fixed limit on how many values a > multi- valued property can at most have. It is currently set to 5. > If you have any such properties with more than 5 values then please > let me know. Is this really necessary? I once had an application that would have used around 30 values in one multi-valued property (project details to be stored on an invoice page - which can cover multiple projects) - I didn't go for multi-valued properties though, since it's impossible to get the data out again in a useful way (which has been brought up a lot of times on these lists, by me and many others). I'm using Semantic Internal Objects now, but as Yaron pointed out, there are issues with it, that would require changes in SMW. I really hope that your recent code changes will finally make it possible to have "bundles of values that belong together" stored in a property (or an internal object, or whatever other thing), and that it's possible to get them out again in a useful way - not introduce new limitations to that feature, that is really necessary as soon as you do something non-trivial with SMW. If there must be a fixed limit, couldn't it be something like 2^14 - like Excel's maximum amount of columns (iirc)? Patrick. -- Key ID: 0x86E346D4http://patrick-nagel.net/key.asc Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4 signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] [News] SVN update warning
On Mittwoch, 6. Januar 2010, zehet...@molgen.mpg.de wrote: > Hi, > > > The change will also introduce some fixed limit on how many values a > > multi- valued property can at most have. It is currently set to 5. If you > > have any such properties with more than 5 values then please let me know. > > If I understand correctly you mean the number of value-parts a multi-valued > property can have. Most of my properties are multivalued and some certainly > have more than just 5 parts. I really hope that 'limit' will be a variable > set via LocalSettings (for my use of SMW this change sounds rather scary > and makes me really kind of nervous). So far, the limit is not planned to be a variable, but we can set a higher value right away if you need it. How many types do your longest n-ary properties have? Cheers, Markus > > Cheers, > Gu > > Quoting Markus Krötzsch : > > In preparation of SMW 1.5, I am currently doing some large-scale changes > > to certain internals of the SMW code. This involves changes in the way in > > which > > > > multi-valued properties (a.k.a. n-ary properties) are handled. > > Eventually, only very little will change from a user's perspective (only > > a small change on > > the property page of each multi-valued properties will be needed), but a > > lot > > > > will change internally. > > > > The change will also introduce some fixed limit on how many values a > > multi- valued property can at most have. It is currently set to 5. If you > > have any such properties with more than 5 values then please let me know. > > > > I therefore suggest to not upgrade wikis that use such properties to the > > development version of SMW until the details on how to modify the pages > > have > > > > been published and everything is known to work again. > > > > I will send a separate email to the developers' list regarding changes > > that might affect developers. > > > > Cheers, > > > > Markus > -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] [News] SVN update warning
Hi, > The change will also introduce some fixed limit on how many values a multi- > valued property can at most have. It is currently set to 5. If you have any > such properties with more than 5 values then please let me know. If I understand correctly you mean the number of value-parts a multi-valued property can have. Most of my properties are multivalued and some certainly have more than just 5 parts. I really hope that 'limit' will be a variable set via LocalSettings (for my use of SMW this change sounds rather scary and makes me really kind of nervous). Cheers, Gu Quoting Markus Krötzsch : > In preparation of SMW 1.5, I am currently doing some large-scale changes to > certain internals of the SMW code. This involves changes in the way in which > > multi-valued properties (a.k.a. n-ary properties) are handled. Eventually, > only very little will change from a user's perspective (only a small change > on > the property page of each multi-valued properties will be needed), but a lot > > will change internally. > > The change will also introduce some fixed limit on how many values a multi- > valued property can at most have. It is currently set to 5. If you have any > such properties with more than 5 values then please let me know. > > I therefore suggest to not upgrade wikis that use such properties to the > development version of SMW until the details on how to modify the pages have > > been published and everything is known to work again. > > I will send a separate email to the developers' list regarding changes that > might affect developers. > > Cheers, > > Markus > > -- > Markus Krötzsch > * Personal page: http://korrekt.org > * Semantic MediaWiki: http://semantic-mediawiki.org > * Semantic Web textbook: http://semantic-web-book.org > -- > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
[SMW-devel] Upcoming changes to SMW code
As announced recently, I am about to do some changes to the internals of SMW. The API of most SMW classes will be stable and almost all extension code is expected to work without changes. This email is to inform developers of SMW- based code about some changes that might be relevant. In particular, this will affect the way in which multi-valued properties are declared and in which values for such properties are represented internally. Currently, properties with multiple values are described by SMWTypesValue objects that hold the list of subtypes. The ID of these types is "__nry". Values for such properties are then represented by SMWNAryValue objects, and queries for such values use SMWValueList objects as descriptions. Your extension could be affected by the changes if your code contains one of the following strings: "__nry", "SMWNAryValue", "SMWValueList", "isUnary". I can add compatibility code that allows code to use (some of) the old functions to some extent. So please let me know if your code uses any of the above, ideally with a pointer to the code, so that I can see what is needed there. Most extensions will not be using any of the above. Cheers, Markus P.S. As I mentioned, I also do significant changes to the storage code. However, this affects neither the API nor the DB layout, so it should not affect extensions. -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
[SMW-devel] [News] SVN update warning
In preparation of SMW 1.5, I am currently doing some large-scale changes to certain internals of the SMW code. This involves changes in the way in which multi-valued properties (a.k.a. n-ary properties) are handled. Eventually, only very little will change from a user's perspective (only a small change on the property page of each multi-valued properties will be needed), but a lot will change internally. The change will also introduce some fixed limit on how many values a multi- valued property can at most have. It is currently set to 5. If you have any such properties with more than 5 values then please let me know. I therefore suggest to not upgrade wikis that use such properties to the development version of SMW until the details on how to modify the pages have been published and everything is known to work again. I will send a separate email to the developers' list regarding changes that might affect developers. Cheers, Markus -- Markus Krötzsch * Personal page: http://korrekt.org * Semantic MediaWiki: http://semantic-mediawiki.org * Semantic Web textbook: http://semantic-web-book.org -- signature.asc Description: This is a digitally signed message part. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
