On 04/08/10 18:23, Hurliman, John wrote:
Does it actually need to hold arbitrary data types, or would the LLSD type
system
> encompass everything you need? It natively handles all the basic C# value
types, the
domain-specific types in OpenMetaverseTypes.dll (vectors/quaternions), plus
arrays and maps.
> The main benefit you get from using an OSDMap is transparent serialization of
arbitrary data.
> There's no way to serialize Dictionary<string, object> transparently* (aside
from digging
> ourselves back into the .NET serialization hole) so you end up with something
like
> Dictionary<string, ISerializableData> and define methods on the
ISerializableData interface for
> storing and loading an OSDMap anyways. Restricting the type system of data
that can be attached to
> SOG/SOPs also makes memory handling and cleanup more straightforward, since
you know there are no
> unmanaged pointers or special resources such as sockets attached directly to
prims that need cleanup
> before prims are thrown away. Finally, it eases dependency issues since you
never have to worry about
> defining a MyExtendedPrimData class in one assembly and trying to typecast to
it in another assembly.
> With over 90 different assemblies in OpenSim right now this can cause major
headaches.
Yes, I started down the path of trying to serialize an IDictionary<string, object> before realizing that this was going
to be impossible without either a) referencing the classes directly in the core [hence defeating the whole purpose] or
b) getting modules to register for keys in the dictionary and invoking them to handle [de]serialization to specific classes.
Option b) starts getting complicated and I'm not sure how well it would work. It could also incur the other costs that
you mention.
So I started implementing [de]serialization of an OSDMap (Attrs) to hold dynamic SOP attributes. In doing this, I have
the following observations/questions.
1) ## Dynamic attributes are best manipulated directly in their OSD form ##. While one could [de]serialize the OSD
into a native class, this incurs both performance penalties and/or makes the code more complicated. It seems easier in
the end just to manipulate OSD directly.
For instance, under this scheme, instead of taking SOP.Attrs["Media"][<face>] and deserializing it to a libomv
MediaEntry class, I would instead get/set variables as something like
SOP.Attrs["Media"][<face>]["current_url"] = new OSDString("http://example.com";);
2) ## Changing OSD attributes is awkward ##. From the example above, you can see that whenever I want to change the
dynamic attribute I need to wrap it in a new OSD class since the code doesn't allow me to change the underlying value.
This is clumsy.
3) ## Stability of OSD attribute names ##. The current_url name above originates from the OSD serialization used for
CAP communication with the client. Could this and other names change? I suspect that any module using OSD would really
have to translate data from the client into its own OSD data structure (even if this was initially identical) in order
to guard against protocol changes.
4) ## OSD/LLSD translates some datatypes (e.g. uint) into byte[] ##. OSD, at least in its LLSD representation, renders
some attributes such as uint into byte[], which harms human readability. I presume this is because it's oriented
towards data transmit rather than storage? This is not ideal for a storage format, at least if one wants some level of
human readability. Perhaps there could be another serialization format that doesn't do these transforms (e.g. stores
uint directly)?
5) ## OSD/LLSD translates some native values into the same datatype ##. For instance, uint, long, ulong and byte[] are
all translated into OSDBinary. This isn't a fatal problem, but it does mean that whoever is using the dynamic attribute
needs to know which native datatype to translate them back into, e.g. OSDBinary.AsLong(), OSDBinary.AsUInteger(), etc.
This is clumsy.
6) ## Attrs locking ##. As OSDMap is a dictionary it would need to be locked on enumeration and hence on update (and
really on read as well on some readings of the Microsoft API docs). This could be awkward unless the OSDMap is wrapped
in a class to lock on access (I already had to wrap it in a class anyway to stop it being auto-serialized).
Even with all these quibbles, I did get as far as implementing dynamic attribute storage for MediaURL and this appeared
to be working fine. However, using the OSD representation of MediaEntry got complicated and so I would rather implement
dynamic attributes separately to reduce complexity. Hence, I've gone back to a conventional implementation of MOAP
SOP/PrimitiveBaseShape attributes.
Sorry for the tl;dr but I think that this is a very interesting area. If we could also save the dynamic attributes to
persistent storage (rather than breaking them up into separate columns) and have a script infrastructure where functions
can be dynamically defined then we could be some of the way to properly separating SL modules and core. Core could be
separated out as a content neutral component rather than as part of an ever-expanding monolith.
Justin
John
-----Original Message-----
From: [email protected] [mailto:opensim-dev-
[email protected]] On Behalf Of Justin Clark-Casey
Sent: Thursday, July 29, 2010 8:55 AM
To: [email protected]
Subject: Re: [Opensim-dev] Proposal: Introduce key:value pair dictionaries
into SOP and PrimitiveBaseShape
On 29/07/10 01:40, Dahlia Trimble wrote:
Similar to the OSDMap in libomv?
Not dissimilar, though the in-memory maps in OpenSim would need to hold
arbitrary data types. We couldn't use OSDMap directly unless we were
happy to serialize objects to and from OSD for every get and set (or change
the values directly in the OSD representation, which might be an idea).
On Wed, Jul 28, 2010 at 2:39 PM, Justin Clark-Casey
<[email protected]<mailto:[email protected]>> wrote:
Hi there. Whilst implementing media-on-a-prim, I've been keeping as
much code in the MOAP region module as possible.
I'm quite impressed with how feasible this is. However, there
remain three major structures where the core of OpenSim has to
understand something about media on a prim.
1) Database plugins - to get/put values to named database columns
(e.g. prims.MediaURL).
2) Script functions (e.g. llGetPrimMediaParams()).
3) Scene objects (PrimitiveBaseShape.Media and
SceneObjectPart.MediaURL).
It's difficult to do anything right now about (1) and (2), but I
believe there is an opportunity to address (3).
What I would like to do is introduce dictionaries into
PrimitiveBaseShape and SceneObjectPart that would supplement
existing fields by storing arbitrary key/value pairs. So instead of
having to hardcode a new MediaURL property on SceneObjectPart I
could instead get/put the data as something like
SceneObjectPart.Values["MediaURL"].
Thus, the dictionaries can act as blackboards for communication
between plugins and modules without the core of OpenSim having to
get involved. I think that this would move us a tiny way towards
our vision of being a generic virtual environment platform and away
from hardcoded Second Life specifics, and make it easier to write
more ambitious region modules without additions to core.
Thoughts?
--
Justin Clark-Casey (justincc)
http://justincc.org
http://twitter.com/justincc
_______________________________________________
Opensim-dev mailing list
[email protected]<mailto:[email protected]>
https://lists.berlios.de/mailman/listinfo/opensim-dev
_______________________________________________
Opensim-dev mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/opensim-dev
--
Justin Clark-Casey (justincc)
http://justincc.org
http://twitter.com/justincc
_______________________________________________
Opensim-dev mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/opensim-dev
_______________________________________________
Opensim-dev mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/opensim-dev
--
Justin Clark-Casey (justincc)
http://justincc.org
http://twitter.com/justincc
_______________________________________________
Opensim-dev mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/opensim-dev
