On 12/13/12 4:17 PM, Gary Martin wrote:
On 13/12/12 13:41, Jure Zitnik wrote:
Hi,
On 12/12/12 5:31 PM, Gary Martin wrote:
On 12/12/12 15:18, Jure Zitnik wrote:
2. Something that we might forgot: What about 3rd party plugin
tables that
reference multiproductized Trac tables?
Will probably need to proclaim these incompatible when more
than one
product is in effect?
Good point. To keep track of records from tables from third party
plugins, this approach doesn't quite work. I would have thought
that we would be better off using a separate table to keep track
of the resources that belong to a product. Is this another area
that has not been updated based on discussions?
The current SQL translator implementation would show 3rd party
plugins a view of translated tables that would only include
resources from the currently selected product scope. If the plugin
makes a reference to a resource by it's name everything should work
fine as the reference would be consistent each time when in that
specific product environment (as the plugin would always get the
same view of the database).
Things start breaking if there's a resource with the same name in
multiple products, unless the translator is changed to return names
with product namespace being prefixed to the actual resource name
for example. The plugins would get version name 'BH:1.0' instead of
'1.0' for example. Still, this doesn't solve the problem entirely
as the plugin (that's not aware of products) would end up (in it's
own tables) with references to different resources from different
products and maybe that's not exactly what's expected to happen...
Keeping track of resource belonging to a product using a separate
resource mapping table also unfortunately doesn't solve the issue.
We'd need to change the schema anyway as in the current database
model, all tables have 'name' column as their key. We could of
course reference the same resource from different products using
the separate mapping table but we'd be referencing the same record
and changing the name of that record would change the resource in
all products which is, at least imo, not what we want.
Ah yes, I forgot about my ideas for that. For the purposes of unique
keys I was thinking of including some kind of prefix as part of the
name - not necessarily the product namespace as we could consider it
better to leave this as a constant with a means to link the prefix
to the namespace.
As I wrote above, only prefixing names doesn't solve the issue. If we
have a product unaware plugin, that plugin should only see resources
from the current product scope and this (for trac/bh resources) is
currently accomplished by translating SQLs in such a way that the
plugin only sees a product specific view of resources. The problem
arises when the plugin stores references to that resources in their
own tables as it might end up (unless we do something about it) with
resource mixed from different products.
Let's say we have a custom table named 'sorted_milestones' with
columns 'milestone_name' and 'sequence' and database constraint that
'sequence' is unique. If plugins sees only resources (in this case
milestones) from one product at a time, constraint will fail as logic
sees only product scope specific part of milestones. When product
scope would change, the plugin sees completely different set of
milestones and has no way of knowing that certain sequences are
already 'taken'.
On the other hand, assuming we only prefix resources without
filtering them for multi-product unaware plugins would cause the user
interface for those plugins to show everything (every resource
defined regardless of product scope). In addition to that, it'd be
hard (if not impossible) to remove the prefix before showing that to
the user. Also, it's not exactly obvious that we can remove the
prefix before passing that to plugins/UI as we never know how those
will be used further down the line...
The problem with just adding fields to each model is not so much a
problem from the point of view of 3rd party plugins accessing those
models that are modified in such a way, but rather with those
resource tables that are added by the third party plugins.
Completely agree.
These would have to be modified to add the product to their tables
too. Is the suggestion that we do that modification for externally
defined resources or only provide the ability for specific plugins?
The idea we are playing with is that in addition to translating 3rd
party plugin DMLs (SELECT/INSERT/UPDATE/DELETE) (to present plugins
with a view of trac/bh resources as seen from the current product
scope), the DDLs (CREATE/ALTER/DROP) for the custom tables should
also be translated to support product scope(s) - this would be
accomplished by creating per product custom table(s), prefixing the
table name with product prefix. This (in combination with the
currently implemented SQL translation) would present product unaware
3rd party plugins with product scoped tables for both, trac/bh tables
and any custom table the plugin would create. References between the
tables would also work and the content of the tables would always
represent data based on the resources in the product scope.
Naive approach would be to solve 3rd party custom table the same way
as trac/bh tables (by adding product column). This does not work for
two reasons:
- schema upgrades - if the plugin chooses to upgrade it's custom
table schema the usual way of doing this is to copy data to temp
table, drop original table (this would effectively drop data for all
products), recreate new table with changed schema and fill it from
temp table
- table schema changes - not really sure how to implement ALTER TABLE
if we modify the original schema (connected with the first reason)
To summarize the idea:
1. for 3rd party plugins that are product unaware, any custom table
being created is namespaced to the product by prefixing the table
name with the product name (in a similar way as discussed resource
name columns). SQL translator functionality will need to be extended
to support DDL.
2. the SQL translator will be changed in such a way that it will
support the following table 'types':
- non-translated tables - tables that need no translation
(session, cache, attachments,...)
- trac/bh tables with product scope - these are tables with
product specific resource (enum, component, ticket, milestone,
version,...) - the product scope for these tables is implemented
using 'product' column
- 3rd party, product-unaware plugin custom tables - product scope
for these tables is implemented by prefixing the table name with
product prefix
3. changes to product unaware plugin install/upgrade process -
install/upgrade (IEnvironmentSetupParticipant) would need to be
invoked for all currently defined products (within that product scope
of course)
4. adding a new product would need to invoke 3rd party (product
unaware) plugin installation
How does that sound to everyone?
Thanks.. we definitely needed this to evaluate the solution properly!
I don't strictly mind what solution is implemented as long as we are
not going too far down a dead end. I would also generally prefer
whichever solution turned out to be simplest but that will be far
harder to judge.
Interestingly, if you are also looking at managing table names, it is
not inconceivable that you could add the product fields to the 3rd
party resource tables after all, as long as you were also prepared to
discover which fields should be unique together with product. One of
the possible dangers with this might be that a plugin is at some point
allowed to adjust the schema (obviously we wouldn't mean to allow it -
it would still be possible though) so it still might be wise to change
the table name anyway. There is the possible advantage that it limits
the differences in processing a little.
That (adding product column to 3rd party resources) was one of the ideas
that were evaluated. The problem with that approach is the schema
upgrade process that usually goes like this (even in trac itself):
1. copy data to temp table
2. drop original
3. create new table with changed schema
4. fill new table from temp table & drop temp table
The problem is step 2 (table drop) as there is no way of preventing
plugin from dropping the table regardless of the fact that the table
holds data for all product scopes and not just the current one.
There is actually one final thing that seems to have been left without
any consideration, and that is whether we can get away with allowing
third party resources to be available across all scopes. I assume that
the initial implementations will start off this way anyway!
If we take the 'table name prefix' approach, 3rd party resource will
only be available in current product scope. Only product aware plugins
and bloodhound code will be able to access resources through all scopes
using the global database connection that will have the SQL translator
turned off. The interesting issue here is how to present that 3rd party
resource (scattered across multiple tables) through global scope without
requiring them to do UNION SELECTs on all tables. We might provide
helper functions for that though ... something to think about but
definitely not critical at the moment.
Cheers,
Jure
