On Wed, Apr 26, 2017 at 3:49 PM, David Adams via 4D_Tech <
4d_tech@lists.4d.com> wrote:
> I just went back to the top of this thread and scanned down...and I think
> that I'm not understanding a key detail. Douglas, you're saying that the
> packed records have 'meta-data', but it sounds like that data is a map to
> the packing. So, packed data types and offsets, something of that sort.
>
"map" is a much better description. The data is "in line" in the BLOB along
with the actual data.
> Would it be possible to re-engineer it so that you have key summary
> information stored somewhere? The ideal way to optimize something slow is
> to not do it at all!
>
Great way to put it.
"re-engineer" - I'm not all that keen on updating this code but, as you
point out, perhaps a small change can yield significant benefit.
It would seem to make far more sense to do something along the lines of,
for example, sending out a longint array where $byteOrder_AL{Type} contains
the Byte order value rather than accessing a wrapper routine for every
field type every time a value is pulled out of the BLOB.
Sheesh, we could even just use the literal value in the code! :-)
> I didn't get a sense what or if that might be. Is
> there a datestamp or s version stamp, or some sort of checksum that you're
> using to figure out if a record needs transmitting? If so, what about
> injecting that into the header, another field, an object field (seems like
> you might have a good use for an object field to serve as a key indicator
> store)? If this would prevent needless unpacking and needless transmission,
> it's potentially a big win. The break-even is that you avoid enough
> unpacking+transmission to pay for the extra storage cost & checks.
> Actually, if it were something searchable, perhaps you could search for the
> records that need sync using a simple index search (or searches) and then
> just bang through the result.
>
> I suspect I'm off the mark here, but just in case...I'm posting.
>
Glad to see you're posting again.
The app is in V13 now and will be moving to V15 over the Summer so there's
no 4D Object available yet.
The code checks each field against Old and, if the field has changed, it's
encoded and bundled into a BLOB. I don't know the algorithm used to crete
additional BLOB's but average number of records updated BLOB is 18 with a
median value of 7 (I've got data on about 700 synch sessions).
In addition to taking the server a while to chew through the records, it's
no picnic for the road warriors. They're on laptops and probably don't have
SSD's (something I didn't think of when I responded to Jim) so what takes
10 minutes on the server takes much longer when the user syncs and has to
unbundle all of these records on their little $1100 Dell laptop.
--
Douglas von Roeder
949-336-2902
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ: http://lists.4d.com/faqnug.html
Archive: http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:4d_tech-unsubscr...@lists.4d.com
**********************************************************************
