Howdy Greg #2 (or 3?) Haven't seen you since the Silverlight weekend in Docklands a few years ago.
Very interesting! You have implemented your own data compression, and we used to do very similar things back in the late 70s and 80s when mainframe disk space was precious. Compression algorithms and software were not available or widely know then. In fact, Wikipedia says the LZ algorithms were only published in 1977/78 (not long ago in coding years). However, I have this uneasy feeling that all of your manual work is made mostly redundant by what zipping does for transmission. Zip will aggressively remove redundancy from your data, so well in fact that I suspect it might reduce the benefits of your pre-processing to a hair's width. Although your pre-processing will save space for the raw data if that's a problem on the client side. I was quite amazed that ~6000 of my entities as XML took 6.06MB, but deflated down to 492KB which is 8% of the original size and quite suitable for transmission as a single blob. I reckon your data as plain CSV would also reduce incredibly well. Given that I also deflate for transmission as a blob, I think my problem is now reduced to a pure coding problem: What format is easiest to round-trip my entities? Importantly, I'm looking for a general purpose way of transforming (most of the) the entity class properties. XML needs manual coding, Json I'm not sure about. I can't use pure binary serialization because it's not supported in Silverlight clients. Greg K
