On Aug 8, 2013, at 14:45 , Paul Davis <paul.joseph.da...@gmail.com> wrote:
> A behavior isn't quite right because as Jan points out, you may not
> want to implement every hook. Thinking briefly on the idea I think the
> general idea is good but I'd implement it via message passing rather
> than callbacks. While in the happy case callbacks are usually faster
> than message passing its fairly easy to get into a situation where a
> particular callback ends up being slow which will slow down the
> calling process (it sounds obvious and it kind of is, but its easy to
> miss places where a function that's usually quick suddenly starts
> taking milliseconds to run which causes the mailbox to explode if it
> can't receive messages fast enough).
>
> A good example of this is how couch_db_update_notifier works with
> gen_event. When a node gets overloaded due to changes feed listeners
> or other random slowdown in this function it can lead to the entire
> system halting because its waiting for events to be processed through
> this function. After having dealt with that its become fairly obvious
> to me that gen_event is only useful when the number of callbacks is a
> constant (or rather, not dependent on user input).
>
> On the one hand plugins using hooks should be a relatively small
> number of well known functions that we can patch if they end up
> causing problems. On the other hand if we ever plan on supporting
> closed source plugins then I'd vote for the safer message passing
> approach as a preventative measure against misbehaving code.
Excellent, thanks Paul!
How would this work if we’d allow a plugin to, say, format the log
string before it goes into the regular log?
Happy to conclude that we don’t support that (maybe in v1), but we’d
be limiting what plugins can do.
As a general warning we should let our users know that plugins can
screw things up unpredictably (at least in a plugin’s early days).
>
> On Thu, Aug 8, 2013 at 7:25 AM, Jan Lehnardt <j...@apache.org> wrote:
>>
>> On Aug 8, 2013, at 14:05 , Jason Smith <j...@apache.org> wrote:
>>
>>> Note, I intentially began this discussion as a question, not a statement.
>>> Totally thinking "aloud."
>>
>> Same here, thanks for exploring this. I do like `on` :)
>>
>> Jan
>> --
>>
>>>
>>> I think it is like gen_server. For gen_server all you need is a
>>> handle_call/3 and handle_cast/2. The real API is inside that.
>>>
>>> For example, maybe all that's required for the behaviour is on/1 and we can
>>> make a flexible hook system from that. (Now why would "on" be a meaningful
>>> or useful function name?)
>>>
>>> on({log, Severity, Message}) -> ok
>>> , io:format("Couch says ~s\n", [Message])
>>> ;
>>>
>>> % Suppose I want a database that stays synced to my config
>>> % (something I personally need at the moment).
>>>
>>> on({startup}) -> ok
>>> , io:format("CouchDB started!\n")
>>> , init_my_config_db(couch_config:get("*/*")) % Pseudocode
>>> ;
>>>
>>> on({config, Section, Key, Value}) -> ok
>>> , io:format("Config update: ~s/~s = ~p\n", [Section, Key, Value])
>>> , update_my_config_db(Section, Key, Value)
>>> ;
>>>
>>> on(Unknown) -> ok
>>> , io:format("Did you know there is a ~p hook?\n", [element(1, Unknown)])
>>> .
>>>
>>>
>>>
>>> On Thu, Aug 8, 2013 at 5:42 PM, Jan Lehnardt <j...@apache.org> wrote:
>>>
>>>>
>>>> On Aug 8, 2013, at 12:39 , Jason Smith <j...@apache.org> wrote:
>>>>
>>>>> Well, I just googled it. Basically there is a couchdb_plugin.erl which
>>>>> tells Erlang what a behavior looks like. And all that does is define the
>>>>> functions and arity which a couchdb_plugin would have to export.
>>>>>
>>>>> Probably there are some better Erlangers on the list who might chime in.
>>>> It
>>>>> looks like okay bang-for-buck; only not much bang or much buck.
>>>>
>>>> how would this work if a plugin is only interested in handling a single
>>>> hook,
>>>> would it have to implement mock funs for all hooks then?
>>>>
>>>>> On Thu, Aug 8, 2013 at 5:26 PM, Jan Lehnardt <j...@apache.org> wrote:
>>>>>
>>>>>> how would this look in code?
>>>>>>
>>>>>> On Aug 8, 2013, at 12:21 , Jason Smith <j...@apache.org> wrote:
>>>>>>
>>>>>>> Perhaps a custom behaviour to help catch API problems at compile time?
>>>>>>>
>>>>>>> -behaviour(couchdb_plugin).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 8, 2013 at 3:47 PM, Jan Lehnardt <j...@apache.org> wrote:
>>>>>>>
>>>>>>>> Heya,
>>>>>>>>
>>>>>>>> I’m toying with the idea of moving some of my experimental into
>>>>>>>> bona-fide plugins. One of them is my log_to_db branch that on top of
>>>>>>>> writing log messages to a text file also writes a document to a log
>>>>>>>> database.
>>>>>>>>
>>>>>>>> Conceptually, this is the perfect use of a plugin: the feature is not
>>>>>>>> useful in the general case, because *any* activity creates write load
>>>>>>>> on a single database, but for certain low-volume installations, this
>>>>>>>> might be a useful feature (I wouldn’t have written it, if I hadn’t
>>>>>>>> needed it at some point) so allowing users to enable it as a plugin
>>>>>>>> would be nice.
>>>>>>>>
>>>>>>>> But regardless of whether my plugin is useful, it illustrates an
>>>>>>>> interesting point:
>>>>>>>>
>>>>>>>> A log_to_db plugin would need to register for logging events or, if it
>>>>>>>> doesn’t want to duplicate all the logging-level logic in couch_log, it
>>>>>>>> would need some way of injecting a function call into
>>>>>>>> `couch_log:log().`. We could of course try and find a way where a
>>>>>>>> plugin would be able to provide an API compatible version of a CouchDB
>>>>>>>> module and swap it out for it’s custom one, but that’s hardly a great
>>>>>>>> idea.
>>>>>>>>
>>>>>>>> Other software has the notion of “hooks” (some may call it something
>>>>>>>> else) where at well defined points in the main code base, external
>>>>>>>> functions get called with certain parameters. To make things dynamic,
>>>>>>>> there might be a way for plugins to register to be called by those
>>>>>>>> hooks and the main code then asks the registry whether there are any
>>>>>>>> plugin functions to call.
>>>>>>>>
>>>>>>>> In the log_to_db example, we’d have something like this:
>>>>>>>>
>>>>>>>> couch_log_to_db.erl:
>>>>>>>>
>>>>>>>> init() ->
>>>>>>>> couch_hooks:register(couch_log_hook, log_hook_fun/1),
>>>>>>>> ok.
>>>>>>>>
>>>>>>>> log_hook_fun(Log) ->
>>>>>>>> % do the log_to_db magic
>>>>>>>> ok.
>>>>>>>>
>>>>>>>>
>>>>>>>> couch_hooks.erl:
>>>>>>>>
>>>>>>>> register(Hook, Fun) ->
>>>>>>>> % store the Fun with the Hook somewhere
>>>>>>>> ok.
>>>>>>>>
>>>>>>>> call(Hook, Args) ->
>>>>>>>> % retrieve Fun for Hook from somewhere
>>>>>>>> Fun(Args).
>>>>>>>>
>>>>>>>> couch_log.erl:
>>>>>>>>
>>>>>>>> % in log()
>>>>>>>>
>>>>>>>> ...
>>>>>>>> couch_hooks:call(couch_log_hook, Args),
>>>>>>>> ...
>>>>>>>>
>>>>>>>> The main code would define what the hook name and arguments are and
>>>> the
>>>>>>>> plugin would have to conform. The plugin registry would just manage
>>>> the
>>>>>>>> registration and calling of functions for a hook, but nothing more.
>>>>>>>>
>>>>>>>> * * *
>>>>>>>>
>>>>>>>> This is just my first stab at this not thinking about it too much and
>>>> I
>>>>>>>> likely miss some subtleties in Erlang that make this not work (hot
>>>> code
>>>>>>>> upgrades e.g.).
>>>>>>>>
>>>>>>>>
>>>>>>>> How do you think we should implement a hooks feature in CouchDB?
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Jan
>>>>>>>> --
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
signature.asc
Description: Message signed with OpenPGP using GPGMail
