Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: jure Type: | Status: closed enhancement| Milestone: Release 6 Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: fixed | model -+- Changes (by olemis): * milestone: = Release 6 -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:19 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: jure Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Changes (by gjm): * owner: gjm = jure Comment: jure - can this be closed yet? re-assigning to you. -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:17 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: jure Type: | Status: closed enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: fixed | model -+- Changes (by olemis): * status: review = closed * resolution: = fixed Comment: I'm closing this ticket based on the fact that the solution has been committed in `bep_0003_multiproduct` branch , all test cases are ok , and further modifications will be tracked in the context of the work made in #355 , and tickets spawned from that point . Please reopen if my reasons are not accurate or sufficient to justify this status change . -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:18 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by jure): Patch t115_r1437383_inherit_product_config.diff applied in r1438016. -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:14 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by jure): Patch t115_r1436300_option_accessor.diff applied in r1438019. -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:15 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by olemis): Replying to [comment:15 jure]: Patch t115_r1436300_option_accessor.diff applied in r1438019. Please notice that [trac:comment:2:ticket:11023 this patch is scheduled for Trac=1.0.2] , so we should keep an eye on it as it might be already there next time we upgrade vendor branch . ;) -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:16 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by olemis): [attachment:t115_r1436300_option_accessor.diff Attached patch] has been proposed to be committed upstream in trac:ticket:11023 . It is related to comment:5:ticket:355 . -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:12 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by jure): Replying to [comment:10 olemis]: I refreshed the patches because I found typos making config test cases fail . Applied in r1433881 -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:11 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by olemis): I refreshed the patches because I found typos making fail config test cases . -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:10 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings
-+-
Reporter: olemis | Owner: gjm
Type: | Status: review
enhancement| Milestone:
Priority: major |Version:
Component: | Keywords: multiproduct environment
multiproduct | configuration database bep-0003 db-upgrade
Resolution: | model
-+-
Comment (by jure):
Replying to [comment:7 olemis]:
[attachment:t115_r1431447_product_envs_bep3_p1.diff Attached patch]
implements a number of enhancements aimed at complying with bep:0003
specification . This includes :
- Type checking in product env initializer
This tests parent against trac.env.Environment. After the hooks are in
place the proper type of the parent environment should be
multiproduct.env.Environment.
I also spent some time to build [https://bitbucket.org/olemis
/bloodhound-
mq/src/t115_bep3_product_env/t115/t115_r1431447_product_envs_bep3_p1.diff?at=t115_bep3_product_env
another patch] rebasing product environments solution against
`bep_0003_,multiproduct` branch .
Great, will try to commit the patch today as tests are passing after
r1432924 (see below).
{{{
#!sh
$ python setup.py test -m tests.env
[...]
Testing env.get_known_users ... ERROR
Testing env.get_version ... ERROR
test_attr_forward_parent (tests.env.ProductEnvApiTestCase) ... ok
test_typecheck (tests.env.ProductEnvApiTestCase) ... ERROR
[...]
--
Ran 4 tests in 7.610s
FAILED (errors=3)
}}}
... so even if solution works it is not quite ready to be committed in
that branch yet .
Tests are now passing after r1432924 ... there was code in Product.insert
that would auto populate product tables (components, milestones, versions,
etc.) upon product insertion. This should actually be performed by a
higher level logic (admin interface for example).
--
Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:8
Apache Bloodhound https://issues.apache.org/bloodhound/
The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings
-+-
Reporter: olemis | Owner: gjm
Type: | Status: review
enhancement| Milestone:
Priority: major |Version:
Component: | Keywords: multiproduct environment
multiproduct | configuration database bep-0003 db-upgrade
Resolution: | model
-+-
Comment (by olemis):
[attachment:t115_r1431447_product_envs_bep3_p1.diff Attached patch]
implements a number of enhancements aimed at complying with bep:0003
specification . This includes :
- Type checking in product env initializer
- Test cases for product env attributes copied from Environment
- Product environments forward attribute access request to parent
environment
- sqlparse as dependency in setup.py
- unittest2 as dependency in setup.py , needed to run the test suite
- compatibility with setuptools 'test' command
- ProductEnvironment attributes : __getattr__ added , env = parent
... and then ...
{{{
#!sh
$ /srv/venv/python/trac/0.13/bin/python setup.py test -m tests.env
running test
running egg_info
writing BloodhoundMultiProduct.egg-info/PKG-INFO
writing top-level names to BloodhoundMultiProduct.egg-info/top_level.txt
writing dependency_links to BloodhoundMultiProduct.egg-
info/dependency_links.txt
writing entry points to BloodhoundMultiProduct.egg-info/entry_points.txt
writing BloodhoundMultiProduct.egg-info/PKG-INFO
writing top-level names to BloodhoundMultiProduct.egg-info/top_level.txt
writing dependency_links to BloodhoundMultiProduct.egg-
info/dependency_links.txt
writing entry points to BloodhoundMultiProduct.egg-info/entry_points.txt
reading manifest file 'BloodhoundMultiProduct.egg-info/SOURCES.txt'
writing manifest file 'BloodhoundMultiProduct.egg-info/SOURCES.txt'
running build_ext
Testing env.get_known_users ... ERROR
Testing env.get_version ... ok
test_attr_forward_parent (tests.env.ProductEnvApiTestCase) ... ok
test_typecheck (tests.env.ProductEnvApiTestCase) ... ok
==
ERROR: Testing env.get_known_users
--
Traceback (most recent call last):
File /path/to/bloodhound/trac/trac/tests/env.py, line 30, in
test_get_known_users
with self.env.db_transaction as db:
File /path/to/bloodhound/bloodhound_multiproduct/multiproduct/env.py,
line 343, in db_transaction
raise NotImplementedError
NotImplementedError
--
Ran 4 tests in 7.218s
FAILED (errors=1)
}}}
I also spent some time to build [https://bitbucket.org/olemis/bloodhound-
mq/src/t115_bep3_product_env/t115/t115_r1431447_product_envs_bep3_p1.diff?at=t115_bep3_product_env
another patch] rebasing product environments solution against
`bep_0003_,multiproduct` branch .
{{{
#!sh
$ python setup.py test -m tests.env
running test
running egg_info
writing BloodhoundMultiProduct.egg-info/PKG-INFO
writing top-level names to BloodhoundMultiProduct.egg-info/top_level.txt
writing dependency_links to BloodhoundMultiProduct.egg-
info/dependency_links.txt
writing entry points to BloodhoundMultiProduct.egg-info/entry_points.txt
writing BloodhoundMultiProduct.egg-info/PKG-INFO
writing top-level names to BloodhoundMultiProduct.egg-info/top_level.txt
writing dependency_links to BloodhoundMultiProduct.egg-
info/dependency_links.txt
writing entry points to BloodhoundMultiProduct.egg-info/entry_points.txt
reading manifest file 'BloodhoundMultiProduct.egg-info/SOURCES.txt'
writing manifest file 'BloodhoundMultiProduct.egg-info/SOURCES.txt'
running build_ext
Testing env.get_known_users ... ERROR
Testing env.get_version ... ERROR
test_attr_forward_parent (tests.env.ProductEnvApiTestCase) ... ok
test_typecheck (tests.env.ProductEnvApiTestCase) ... ERROR
==
ERROR: Testing env.get_known_users
--
Traceback (most recent call last):
File /path/to/bloodhound/bloodhound_multiproduct/tests/env.py, line
133, in setUp
self._load_product_from_data(self.global_env, self.default_product)
File /path/to/bloodhound/bloodhound_multiproduct/tests/env.py, line
99, in _load_product_from_data
product.insert()
File
/path/to/bloodhound/bloodhound_multiproduct/multiproduct/model.py, line
98, in insert
','.join(['%s'] * len(cols))), vals)
File /path/to/bloodhound/trac/trac/db/util.py, line 139, in
executemany
cursor.executemany(query, params)
File /path/to/bloodhound/trac/trac/db/util.py, line 85, in executemany
return self.cursor.executemany(sql_escape_percent(sql), args)
File
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings
-+-
Reporter: olemis | Owner: gjm
Type: | Status: review
enhancement| Milestone:
Priority: major |Version:
Component: | Keywords: multiproduct environment
multiproduct | configuration database bep-0003 db-upgrade
Resolution: | model
-+-
Changes (by olemis):
* keywords: multiproduct environment configuration database bep-0003 =
multiproduct environment configuration database bep-0003 db-upgrade
model
* owner: olemis = gjm
* status: accepted = review
Comment:
[https://bitbucket.org/olemis/bloodhound-
mq/src/t115_product_env/t115?at=05a393c98a6a Patches @ r05a393c98a6a in
t115_product_env branch] attached to this ticket . They should be applied
in the following [https://bitbucket.org/olemis/bloodhound-
mq/src/t115_product_env/series?at=05a393c98a6a order] against r1429886 .
{{{
#!sh
$ hg qseries
t115/t115_r1429886_product_envs.diff
t115/t115_r1429886_product_config.diff
t115/t115_r1429886_product_envs_testing.diff
}}}
They add product environments , product-specific configuration and test
cases for both features , in that order . These are the test results.
{{{
#!sh
$ python bloodhound_multiproduct/tests/config.py
...
--
Ran 27 tests in 2.474s
OK
}}}
... so IMO they are ready for review .
PS: Environment upgrade is required to add product settings table.
--
Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:5
Apache Bloodhound https://issues.apache.org/bloodhound/
The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: gjm Type: | Status: review enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 db-upgrade Resolution: | model -+- Comment (by olemis): Replying to [comment:5 olemis]: [...] They add product environments , product-specific configuration and test cases for both features , in that order . These are the test results. [...] btw, it's important to mention that so far I've been just focused on rewritting Trac tests to run them against product envs . We'll need a few more to assert product-specific behaviors . The purpose of rewriting Trac tests is to have a idea of how much compatible they will be , and be aware of how much transparent will be the migration towards product envs for plugins. Nonetheless , we'll see the big picture once [ticket:288 DB proxies] will be definitely installed in product envs . So far DB-related methods just raise `NotImplementedError` . -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:6 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings -+- Reporter: olemis | Owner: olemis Type: | Status: accepted enhancement| Milestone: Priority: major |Version: Component: | Keywords: multiproduct environment multiproduct | configuration database bep-0003 Resolution: | -+- Comment (by olemis): I'll be developing this in [https://bitbucket.org/olemis/bloodhound-mq my patch queue @ Bitbucket] in [https://bitbucket.org/olemis/bloodhound- mq/commits/all/tip/branch(%22t115_product_env%22) branch t115_product_env] -- Ticket URL: https://issues.apache.org/bloodhound/ticket/115#comment:4 Apache Bloodhound https://issues.apache.org/bloodhound/ The Apache Bloodhound (incubating) issue tracker
Re: [Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings
-+-
Reporter: olemis | Owner: olemis
Type: | Status: accepted
enhancement| Milestone:
Priority: major |Version:
Component: | Keywords: multiproduct environment
multiproduct | configuration database bep-0003
Resolution: |
-+-
Changes (by olemis):
* keywords: multiproduct environment configuration database =
multiproduct environment configuration database bep-0003
* owner: nobody = olemis
* status: new = accepted
Old description:
IMO one of the goals of multiproduct plugin should be to behave like
(i.e. not an exact clone though still similar to) an implementation of
former multi-environment setup e.g. via `TRAC_ENV_PARENT_DIR` but inside
a single environment . Hence each product should have its own
configuration settings . For practical reasons maybe it's a good decision
to merge product-specific options with environment's global options. The
former should override the later in a way similar to
[http://trac.edgewall.org/wiki/TracIni#GlobalConfiguration global
configuration] (i.e. `inherit.file` config option) .
Some use cases are :
- Notifications for different products sent to separate mailing lists
(i.e. `notification` section)
- Different default values for ticket fields (i.e. `ticket.default_*`)
- Different workflows for each product
- Product svn repostiory having particular branch and tags (i.e. `svn`
section)
- Product-specific fine grained permissions (i.e.
`authz_policy.authz_file`)
- Different sources and hosts for growl notifications (i.e. `growl`
section
used by [http://trac-hacks.org/wiki/GrowlPlugin GrowlPlugin])
- Product-specific header and logo (i.e. `header_logo` section)
- Different ways to collect statistics on groups of tickets for display
in
the milestone views (i.e. `milestone` section)
Implementation notes
Firstly it'd be nice to store product specific configuration in the
database (though it is possible to reuse existing implementation and load
values from plain-old files e.g.
''/path/to/env/config/product-prefix.ini'' ... = this approach is
cheap enough to be taken into account ''';)''' . Table structure for this
should be pretty straightforward and simple . My initial suggestion is :
{{{
#!py
class ProductSettings(ModelBase):
Product settings table
_meta = {'table_name':'bloodhound_product_settings',
'object_name':'ProductSettings',
'key_fields':['product', 'section', 'option'],
'non_key_fields':['value'],
'no_change_fields':['product', 'section', 'option'],
'unique_fields':[],
}
}}}
All this should happen transparently , so that e.g. plugins source code
will work under these circumstances without requiring any change . In
order to get this done I suggest to inject (at a given time TBD during
request dispatching , filtering , handling , ... ) a replacement (i.e.
proxy object) of Trac environment wrapping the real environment but
returning product-specific configuration object for its `config` field .
Therefore a class implementing `ConfigParser` ''API'' on top of this
database structure is also needed .
Some side-effects may lead to changing current multiproduct
implementation . For instance, product name and description may be
stored/retrieved by referring to product-specific `project.name` and
`project.descr` options respectively , and that will work transparently
due to the underlying magic of `trac.config.Option` descriptor and its
subclasses.
New description:
IMO one of the goals of multiproduct plugin should be to behave like (i.e.
not an exact clone though still similar to) an implementation of former
multi-environment setup e.g. via `TRAC_ENV_PARENT_DIR` but inside a single
environment (i.e. [bep:0003#product_envs product environments] ) . Hence
each product should have its own configuration settings . For practical
reasons maybe it's a good decision to merge product-specific options with
environment's global options. The former should override the later in a
way similar to [http://trac.edgewall.org/wiki/TracIni#GlobalConfiguration
global configuration] (i.e. `inherit.file` config option) .
Some use cases are :
- Notifications for different products sent to separate mailing lists
(i.e. `notification` section)
- Different default values for ticket fields (i.e. `ticket.default_*`)
- Different workflows for each product
- Product svn repostiory having particular branch and tags (i.e. `svn`
section)
- Product-specific fine grained permissions (i.e.
`authz_policy.authz_file`)
- Different sources and hosts for growl notifications (i.e. `growl`
section
used
[Apache Bloodhound] #115: Product-specific settings
#115: Product-specific settings
-+-
Reporter: olemis | Owner: nobody
Type: enhancement | Status: new
Priority: major| Milestone:
Component: multiproduct |Version:
Keywords: multiproduct environment |
configuration database |
-+-
IMO one of the goals of multiproduct plugin should be to behave like (i.e.
not an exact clone though still similar to) an implementation of former
multi-environment setup e.g. via `TRAC_ENV_PARENT_DIR` but inside a single
environment . Hence each product should have its own configuration
settings . For practical reasons maybe it's a good decision to merge
product-specific options with environment's global options. The former
should override the later in a way similar to
[http://trac.edgewall.org/wiki/TracIni#GlobalConfiguration global
configuration] (i.e. `inherit.file` config option) .
Some use cases are :
- Notifications for different products sent to separate mailing lists
(i.e. `notification` section)
- Different default values for ticket fields (i.e. `ticket.default_*`)
- Different workflows for each product
- Product svn repostiory having particular branch and tags (i.e. `svn`
section)
- Project-specific fine grained permissions (i.e.
`authz_policy.authz_file`)
- Different sources and hosts for growl notifications (i.e. `growl`
section
used by [http://trac-hacks.org/wiki/GrowlPlugin GrowlPlugin])
- Project-specific header and logo (i.e. `header_logo` section)
- Different ways to collect statistics on groups of tickets for display
in
the milestone views (i.e. `milestone` section)
Implementation notes
Firstly it'd be nice to store project specific configuration in the
database (though it is possible to reuse existing implementation and load
values from plain-old files e.g.
''/path/to/env/config/project-prefix.ini'' ... = this approach is cheap
enough to be taken into account ''';)''' . Table structure for this should
be pretty straightforward and simple . My initial suggestion is :
{{{
#!python
class ProductSettings(ModelBase):
Product settings table
_meta = {'table_name':'bloodhound_product_settings',
'object_name':'ProductSettings',
'key_fields':['product', 'section', 'option'],
'non_key_fields':['value'],
'no_change_fields':['product', 'section', 'option'],
'unique_fields':[],
}
}}}
All this should happen transparently , so that e.g. plugins source code
will work under these circumstances without requiring any change . In
order to get this done I suggest to inject (at a given time TBD during
request dispatching , filtering , handling , ... ) a replacement (i.e.
proxy object) of Trac environment wrapping the real environment but
returning product-specific configuration object for its `config` field .
Therefore a class implementing `ConfigParser` ''API'' on top of this
database structure is also needed .
Some side-effects may lead to changing current multiproduct implementation
. For instance, product name and description may be stored/retrieved by
referring to product-specific `project.name` and `project.descr` options
respectively , and that will work transparently due to the underlying
magic of `trac.config.Option` descriptor and its subclasses.
--
Ticket URL: https://issues.apache.org/bloodhound/ticket/115
Apache Bloodhound https://issues.apache.org/bloodhound/
The Apache Bloodhound (incubating) issue tracker
