There is a Thrift and C++ API change that we're planning to include in the
upcoming 0.9.5.0 "pre4" release which I describe below. Please respond if
you feel that this change will be problematic.
Currently the C++ Client object includes a Table cache which maps table name
to a smart pointer to a table object. An entry gets created when a table is
first opened and subsequent Namespace::open_table() calls will returned the
cached pointer to the previously created table object. The reason for this
cache is that creating the Table object is fairly expensive.
There are a couple of problems with this caching related to the
ThriftBroker. The first problem has to do with ALTER TABLE. If a table is
opened via the ThriftBroker and then altered through the hypertable shell,
subsequent table access (scanner or mutator) will fail with a
SCHEMA_GENERATION_MISMATCH error because the ThriftBroker's hypertable
client will use the stale cached Table that was created with the previous
schema generation. The second problem occurs when a table is opened via the
ThriftBroker and then it is dropped and re-created via the hypertable shell
(e.g. to clear the table contents). Subsequent access to the table will
fail with TABLE_NOT_FOUND because when a table gets re-created, it gets
assigned a new table ID and the ThriftBroker's hypertable client will use
the stale cached Table that was created with the old table ID. The current
code attempts to solve this problem with the following approaches:
1. 'Hypertable.Client.RefreshSchema' property. This property tells the
Hypertable client to refresh the table cache when it encounters certain
errors (default is "true").
2. 'retry_table_not_found' parameter of the open_scanner API.
3. 'force' flag of the Namespace::open_table() API
This approach does not cover all cases and is somewhat awkward. We propose
the following solution to clean up the API and make the system more
user-friendly:
1. Remove the Hypertable.Client.RefreshSchema property and
the retry_table_not_found parameter to open_scanner API
2. Replace the 'force' flag of Namespace::open_table() with a 'flags'
parameter that is the bitwise OR of the following (defined in Table.h):
enum {
OPEN_FLAG_BYPASS_TABLE_CACHE = 0x01,
OPEN_FLAG_REFRESH_TABLE_CACHE = 0x02,
OPEN_FLAG_REFRESH_SCHEMA_TRANSPARENTLY = 0x04
};
By default, both the ThriftBroker and the hypertable shell will open tables
with the OPEN_FLAG_REFRESH_SCHEMA_TRANSPARENTLY flag. This will cause the
cached Tables to get refreshed automatically (and transparently) when the
client encounters SCHEMA_GENERATION_MISMATCH or TABLE_NOT_FOUND errors. It
has been our experience that this behavior is what most people expect. If
this change will be problematic for you, please let us know (soon) and we
can try to come up with a solution to accomodate.
- Doug
--
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en.
