Hi, Kenton!
> I suppose the first question is: Does your database actually need to > understand the dynamic fields at all? Or does it just need to be able to > echo them back to the client later on, with only the client actually > understanding them? > Yes. It needs to be able to index them and to process queries that reference them as (e.g.) select criteria. While this can (mostly) be done with a single JSON column, the virtual columns required effectively amount to creating new DB columns anyway, at which point it isn't clear why the JSON column is providing benefit when used for this purpose. > If the server needs to understand the fields -- perhaps, to index them, or > something -- then one possibility is the client could upload its schema to > the server in capnp schema format (schema.capnp). On the server, you could > use the "dynamic API" (in C++, capnp/dynamic.h; some other implementations > support it as well but not all) to load this schema dynamically and > introspect the messages based on it. > I had not known about the dynamic API, and I will look in to that. In the application at hand, the mechanism for adding fields is via the API. Both the server and the client know about the resulting change, so I would think that *either* side could send the revised schema to the other. I'm also having disgusting thoughts involving wasm, on-the-fly codegen, and orphaned baby ducks. :-) An advantage to the list of union of tagged value types approach I mentioned might be that all current implementations know how to handle that. If it's changing dynamically, we're going to have to interpret things in any case (at least at *some* level of abstraction). These fields are important, but generally few in number. > Alternatively, you could of course just have the client send names and > values, as you suggest. Though at that point you might even consider just > using JSON (or one of the myriad binary alternative encodings of JSON) as > the Cap'n Proto encoding may actually be adding more bloat than it saves. > Every time that comes up I think unpleasant thoughts about JavaScript and integers. Brendan's decision to represent all numbers as floats got him an implementation in three days, but every one of us has been paying for it every day since then. Even Brendan recognizes that this was an unfortunate call in hindsight. > It's hard to say what's the best approach without really understanding the > details of the application, though. > Here's an attempt at expansion. Suppose you are building a production management application. Part of that is prioritizing what to build. The prioritization depends in part on physical aspects of the products - both for the build itself and for efficiency in gathering components. In consequence, there are fields needed in the item descriptions that would be relegated to a text description in a more conventional inventory system (e.g. for tee shirts: (size, color, type) would usually be implicit in the SKU and described in the description, but for production we need to sort and filter on those for inventory picking). Since you can't anticipate all of the products that will come along, it becomes necessary to let the customer deploying the management system figure out what their additional fields need to be. And just to make matters fun, those fields vary from product to product, which has a way of turning the entire thing into an open union join. Which in turn has unpleasant consequences for implementing updates. But that's a whole different issue. The saving grace, such as it is, is that a given production flow is generally going to sort/select using the same criteria for all of the items it produces. Jonathan -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJdcQk1pUdzj8AO-gFKgxyWoCPTGgEBDGL3Jpz9MWRuz2bO_ew%40mail.gmail.com.
