Re: Do we need 2 entry points for the replication?
Summary:
- Responses seem to indicate that most agree two endpoints is
unnecessarily confusing.
- Robert suggested {persistent: true} or similar.
I propose we add just such a feature.
With a persistent flag and the ability to cancel/delete replications
via /_replicate is there anything else missing?
_replicator could still be available as an admin interface (like how
internal config in many RDMS is in tables) but documentation should
reflect /_replicate as a preferred API, with a clear indication that
the format of replication documents may change at any time (or even
their status as documents).
-Randall
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 11:46 PM, Jason Smith j...@iriscouch.com wrote: On Fri, Jan 20, 2012 at 9:04 AM, Randall Leeds randall.le...@gmail.com wrote: Exposing features as manipulations to normal documents makes CouchDB's API simpler and more orthogonal. On the other hand, exposing features via special-purpose APIs hides the implementation and frees us to change how it works under the hood. Thank you for not saying under the covers. Activity under the hood--or bonnet--hardly resembles activity under the covers. All agree that state is stored in a database. So the question is, have a database and an API defined as changes to it (perhaps via _show, _list, and _update); or, have a database and an API defined otherwise. Either way, you have to bite the bullet and make a breaking change; so is hiding the implementation a different matter? +1 for hiding the implementation. Who says that those other API endpoints that you would use to manage the replication db will be around in a year? We protect ourselves from the future by hiding the implementation details, thereby not making the same mistake twice. Cheers, -- Sam Bisbee
Re: Do we need 2 entry points for the replication?
On 19 January 2012 22:56, Paul Davis paul.joseph.da...@gmail.com wrote: On Thu, Jan 19, 2012 at 3:52 PM, Robert Newson rnew...@apache.org wrote: I would prefer to see a single /_replicate entrypoint, with, say, persistent:true to indicate that the replication settings should be stored. We would also need an API to list all persistent replication tasks and one to delete them. Which would look a lot like the _replicator database, though much more controlled (no public passwords for those jobs that require auth). +1 while I understand *why* we have different APIs at the moment, it is definitely confusing for people. I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. We can break it eventually and I think we should consider it sooner rather than later. As Damien observed, sometimes its not until you tried already that you understand the requirements better. ATM we are concerned that the short-term user impact of reverting an API decision is greater than the damage done long-term by having something that *every* future user will struggle with. Either we release new APIs more frequently, perhaps with an experimental tag, to get real-world feedback, or we need to batch these things up for a larger 2.0 release where we revert a lot of existing functionality. Are there other options that help get features to users quickly, and enable tidying up in future? Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. I seem to recall someone else with a similar opinion even when these things were being designed. ;) Also, what kind of crazy sailing trains do you brits have over there and how do I get a ticket to ride on one? http://farm4.staticflickr.com/3008/5840702274_bd17fe8dee_z.jpg A+ Dave
Re: Do we need 2 entry points for the replication?
I would prefer to see a single /_replicate entrypoint, with, say, persistent:true to indicate that the replication settings should be stored. We would also need an API to list all persistent replication tasks and one to delete them. Which would look a lot like the _replicator database, though much more controlled (no public passwords for those jobs that require auth). I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. B. On 19 January 2012 21:45, Benoit Chesneau bchesn...@gmail.com wrote: Think it's time to relaunch this threads following the 2 separate discussions that we had this morning. Actually we have 2 ways to handle the replication: - `_replicate` : which isn't persistent and where you can follow the task in active tasks - `_replicator`: wich is a plain db where any replication task is persistent. In that case even if a replication task is finished, a document stay in the replicator db and you will have to delete it and compact the db from time to time. Both are their use cases, and while i think it's good to keep the different approaches (persisten against fire and forget), I think the API should be more consistent by offering only 1 end point for the replication. We could then having different parameters depending on the replication type we want (persistent or fire and forget). Also both should appear in the active tasks (maybe this point have been solved since). So I propose to keep only one entry point : _replicate and pass the parameter we want to it. What do you think ? - benoit
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 3:52 PM, Robert Newson rnew...@apache.org wrote: I would prefer to see a single /_replicate entrypoint, with, say, persistent:true to indicate that the replication settings should be stored. We would also need an API to list all persistent replication tasks and one to delete them. Which would look a lot like the _replicator database, though much more controlled (no public passwords for those jobs that require auth). +1 I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. We can break it eventually and I think we should consider it sooner rather than later. Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. I seem to recall someone else with a similar opinion even when these things were being designed. ;) Also, what kind of crazy sailing trains do you brits have over there and how do I get a ticket to ride on one? B. On 19 January 2012 21:45, Benoit Chesneau bchesn...@gmail.com wrote: Think it's time to relaunch this threads following the 2 separate discussions that we had this morning. Actually we have 2 ways to handle the replication: - `_replicate` : which isn't persistent and where you can follow the task in active tasks - `_replicator`: wich is a plain db where any replication task is persistent. In that case even if a replication task is finished, a document stay in the replicator db and you will have to delete it and compact the db from time to time. Both are their use cases, and while i think it's good to keep the different approaches (persisten against fire and forget), I think the API should be more consistent by offering only 1 end point for the replication. We could then having different parameters depending on the replication type we want (persistent or fire and forget). Also both should appear in the active tasks (maybe this point have been solved since). So I propose to keep only one entry point : _replicate and pass the parameter we want to it. What do you think ? - benoit
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 1:52 PM, Robert Newson rnew...@apache.org wrote: I would prefer to see a single /_replicate entrypoint, with, say, persistent:true to indicate that the replication settings should be stored. We would also need an API to list all persistent replication tasks and one to delete them. Which would look a lot like the _replicator database, though much more controlled (no public passwords for those jobs that require auth). +1 I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. Even if it needs another thread, can you elaborate on that? I had started some work for http://markmail.org/message/hogagl2qbhfgxmf7 and I'm curious to know the problems you see in. Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. Imo such dbs should be hidden, and we could only provide entry points accessing to them. For example having all systems dbs under /couchdb/ prefix and hide them for others excepts admins. Just like postgres does. I'm agree it was a mistake to expose them. I think we should stop to do it in near future imo. - benoit
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 1:56 PM, Paul Davis paul.joseph.da...@gmail.com wrote: I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. We can break it eventually and I think we should consider it sooner rather than later. +1
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 14:14, Benoit Chesneau bchesn...@gmail.com wrote: On Thu, Jan 19, 2012 at 1:56 PM, Paul Davis paul.joseph.da...@gmail.com wrote: I think it's too late, though. There's work on master to fix the issues with _replicator now (and the similar ones in _user). While I don't like the approach, it does solve the problem. We can break it eventually and I think we should consider it sooner rather than later. +1 +1 Should we start a list of things we want to break? We're going to have to break some view API stuff for the next SpiderMonkey. Might as well do as much of it at once as possible. I've got a great idea for how to let people know: we'll call it 2.0. It's sexy and I heard The Web got it a few years ago. -R
Re: Do we need 2 entry points for the replication?
On Fri, Jan 20, 2012 at 4:59 AM, Benoit Chesneau bchesn...@gmail.com wrote: Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. Imo such dbs should be hidden, and we could only provide entry points accessing to them. For example having all systems dbs under /couchdb/ prefix and hide them for others excepts admins. Just like postgres does. I'm agree it was a mistake to expose them. I think we should stop to do it in near future imo. Exposing features as manipulations to normal documents makes CouchDB's API simpler and more orthogonal. -- Iris Couch
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 16:59, Jason Smith j...@iriscouch.com wrote: On Fri, Jan 20, 2012 at 4:59 AM, Benoit Chesneau bchesn...@gmail.com wrote: Bottom line: It's my opinion that _replicator (and _user) were wrongly exposed as full-blooded databases when all we needed to use was the database format (and carefully curate API endpoints). But, alas, that train has sailed. Imo such dbs should be hidden, and we could only provide entry points accessing to them. For example having all systems dbs under /couchdb/ prefix and hide them for others excepts admins. Just like postgres does. I'm agree it was a mistake to expose them. I think we should stop to do it in near future imo. Exposing features as manipulations to normal documents makes CouchDB's API simpler and more orthogonal. On the other hand, exposing features via special-purpose APIs hides the implementation and frees us to change how it works under the hood. -- Iris Couch
Re: Do we need 2 entry points for the replication?
On Fri, Jan 20, 2012 at 9:04 AM, Randall Leeds randall.le...@gmail.com wrote: Exposing features as manipulations to normal documents makes CouchDB's API simpler and more orthogonal. On the other hand, exposing features via special-purpose APIs hides the implementation and frees us to change how it works under the hood. Thank you for not saying under the covers. Activity under the hood--or bonnet--hardly resembles activity under the covers. All agree that state is stored in a database. So the question is, have a database and an API defined as changes to it (perhaps via _show, _list, and _update); or, have a database and an API defined otherwise. Either way, you have to bite the bullet and make a breaking change; so is hiding the implementation a different matter? -- Iris Couch
