[jira] [Resolved] (COUCHDB-1412) Temporary Views heading nested incorrectly in doc wiki
[ https://issues.apache.org/jira/browse/COUCHDB-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Cottlehuber resolved COUCHDB-1412. --- Resolution: Fixed Assignee: Dave Cottlehuber added wiki user but can't see incorrect heading myself. Over to you Nathan thanks! Temporary Views heading nested incorrectly in doc wiki Key: COUCHDB-1412 URL: https://issues.apache.org/jira/browse/COUCHDB-1412 Project: CouchDB Issue Type: Bug Components: Documentation Reporter: Nathan Vander Wilt Assignee: Dave Cottlehuber Priority: Trivial Original Estimate: 0.25m Remaining Estimate: 0.25m The View Compaction heading is one level too deep in the wiki: http://wiki.apache.org/couchdb/HTTP_view_API#View_Compaction It appears as a subsection of View Compaction and doesn't appear in the top TOC widget. Easy fix for someone, or I'd be happy to get contrib access to the wiki (NathanVanderWilt). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COUCHDB-1412) Temporary Views heading nested incorrectly in doc wiki
[ https://issues.apache.org/jira/browse/COUCHDB-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Cottlehuber closed COUCHDB-1412. - Temporary Views heading nested incorrectly in doc wiki Key: COUCHDB-1412 URL: https://issues.apache.org/jira/browse/COUCHDB-1412 Project: CouchDB Issue Type: Bug Components: Documentation Reporter: Nathan Vander Wilt Assignee: Dave Cottlehuber Priority: Trivial Original Estimate: 0.25m Remaining Estimate: 0.25m The View Compaction heading is one level too deep in the wiki: http://wiki.apache.org/couchdb/HTTP_view_API#View_Compaction It appears as a subsection of View Compaction and doesn't appear in the top TOC widget. Easy fix for someone, or I'd be happy to get contrib access to the wiki (NathanVanderWilt). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking the 1.2.0 release
On Feb 15, 2012, at 01:41 , Jason Smith wrote:
Thanks for your feedback, Jan.
I am in the camp that views should be allowed but it's not the end of
the world. I would have spoken up but the RFC was security-related and
not public (which is fine, that's life.) I noticed it in Git but
didn't speak up then either.
On Wed, Feb 15, 2012 at 2:51 AM, Jan Lehnardt j...@apache.org wrote:
The reasoning was as follows:
- I tried to design the system to be as simple as possible.
- I also tried to design it to be the least intrusive to the 1.2.x
codebase given the maturity of the branch.
That's cool. However that goal somewhat conflicts with being intrusive
to *users* upgrading to 1.2.x. Specifically, people who used the
CouchApp, Evently, etc. examples and tutorials which use /_users for
public profiles. (The database is is named _users, not _passwords.)
Yup, that is the trade-off. Making the _users db more secure means
taking away existing features. I believe that this is the right thing
to do. :)
It was these goals and J Chris's email about how to solve the conundrum
of per-doc-auth* a few weeks earlier (disable views when you enable
per-doc-auth) that lead to the current situation.
Strong point. However this implementation doesn't meet Chris's
description because we have neither (a) per-doc control, nor (b)
views. If we had per-doc control, then we could allow access to docs
like profile:jhs and disallow org.couchdb.user:jhs.
The system-db security handler for both _users and _replicator disallows
access to documents of user A by user B. That is per-doc-auth in my book.
It isn't user-accessible as it was discussed (and not solved) in the
past, but it is most definitely an implementation of the same concept.
Without (a) or (b), users will have to migrate profiles from /_users
to e.g. /profiles. They can't even replicate, because the point is
protecting the password fields. That's not earth-shattering, but if
you read the BreakingChanges page, you might agree that this is the
biggest user-facing change for a long time.
Yes, and that is for now intentional. :)
My suggestion was a prefab view, /_users/_auth/_view/profiles that
outputs emits public information. Maybe the ddoc also has
.views.lib.profile_config where options are. The map function would
look like this:
function(doc) {
var config = require('views/lib/profiles_config')
if(!config.enabled)
return
if(!doc.public)
return
var val = {}
config.whitelist.forEach(function(key) {
if(key in doc)
val[key] = doc[key]
})
emit(doc._id, val)
}
Thus to have profiles, the admin must:
1. Add a .public = true key to the user doc.
2. Edit .views.lib.profiles_config and set enabled = true
3. Edit .views.lib.profiles_config and add some fields to the
whitelist (e.g. [gravatar_url])
Do you want to put this into a JIRA for 1.2.1 and 1.3.0?
Cheers
Jan
--
Re: Roadmap for the 1.3.0 release
On Feb 15, 2012, at 03:22 , Brian Mitchell wrote: Has there been any discussion around BigCouch integration strategies? It seems like it would fit the bill for the next undertaking on the general couch side. Does anyone from Cloudant have a suggestion for the timeline here? Cloudant havent't yet approached Apache CouchDB with anything regarding their announcement. Any other work from mobile builds and the like might be interesting to support. Were there any interesting changes to pull in from the mobile and embedded device ports? Also, I'm not sure where the view engine refactoring work ended up… I'll look at JIRA but maybe there are specific follow-ups to be made here. The biggest one IIRC was the use of emonk instead of an external Spidermonkey for iOS. Maybe we can make that a compile option like ./configure --enable-emonk-view-server so people can choose which one to run. Finally, for our included javascript libraries and Futon runtime, are we going to replace or update anything here? I'd imagine a newer jQuery could be an advantage for those doing CouchApps. Good that you bring this up. There's also other upstream dependencies that we should look at updating and we don't have a process for that, it usually happens whenever a developer feels like it. I haven't thought about this much yet, but maybe update all dependencies could be a todo before a release branch is created? Cheers Jan --
[jira] [Commented] (COUCHDB-1398) improve view filtering in changes
[
https://issues.apache.org/jira/browse/COUCHDB-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208441#comment-13208441
]
Benoit Chesneau commented on COUCHDB-1398:
--
Hi Filipe,
Thanks for the review, I will address issues later. Just quick comments :
1) I will fix that. I prefer this indentation to the other since it distinct
the fun from other level. But will comply with common style, np :)
2) I thought opening the ddoc each time was enough to handle the ddoc changes.
Isn't raising an exception enough to stop the feed? But of course checking if
the ddoc has changed may be better at this point. I will have a look. Also
errors should be probably catched.
3) In fact I'm eagerly waiting for an event hub in couch. Where you can publish
any event and subscribe too it. I started to use couch_event from @davisp but
stopped at some point since I wasn't sure it was the correct solution. To
minimize the code base, I just reuse the current event handler. I've no strong
opinion on it. Do you think it's really needed?
4) Manual replications tests worked. The `_view` filter is already tested. I
will add tests to address cases where you already have doc, when ddoc disapear
such.
Again thanks for the review.
improve view filtering in changes
-
Key: COUCHDB-1398
URL: https://issues.apache.org/jira/browse/COUCHDB-1398
Project: CouchDB
Issue Type: Improvement
Components: View Server Support
Affects Versions: 2.0, 1.3
Reporter: Benoit Chesneau
Labels: changes, view
Attachments: 0001-white-spaces.patch,
0002-initial-step-move-the-code-from-couch_httpd_db-to-co.patch,
0003-fix-indent.patch,
0004-This-wrapper-is-useless-somehow-split-the-code-in-a-.patch,
0005-add-view_updated-event.patch, 0006-immprove-view-filter.patch,
0007-useless-info.patch, 0008-whitespaces.patch,
0009-handle-native-filters-in-views.patch
Improve the native view filter `_view` support by really using view index.
This patches add following features
- small refactoring: create the couch_httpd_changes modules, to put all the
changes http support in its own module instead having it in couch_httpd_db.
- add the `view_updated` event when a view index is updated : {view_updated,
{DbName, IndexName}}
- start the feed using results in the view index instead of all the db index
- only react on view index changes.
For now next changes are still get using the couch_db:changes_since function
and passing the map function to the results. It could be improved if we had a
by_seq btree in the view index too. Other way may be to skip a number of the
documents already processed. Not sure it would be faster. Thoughts ?
The branch couch_view_changes in my repo contains preliminary support:
https://github.com/benoitc/couchdb/tree/couch_view_changes
Diff:
https://github.com/benoitc/couchdb/compare/master...couch_view_changes
To use it, use the native filter named _view which take the parameter
view=DesignName/Viewname
eg:
http://server/db/_changes?feed=continuousheartbeat=truefilter=_viewview=DesignName/SomeView
It has also an interresting side effect: on each db updates the view index
refresh is triggered so view updates are triggered. Maybe we could introduce
an optionnal parameter to not trigger them though?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Roadmap for the 1.3.0 release
On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote: Has there been any discussion around BigCouch integration strategies? It seems like it would fit the bill for the next undertaking on the general couch side. Does anyone from Cloudant have a suggestion for the timeline here? There's been a lot of discussion in #couchdb and #couchdb-dev but little on the ml. I'm not sure about timeline. There seems to be a lot of issues, most of them minor technical ones (the type that readily bog down once more than 3 people get involved). BigCouch embeds couchdb and was architected to be the clustering layer that couchdb lacks, so in that sense I think we're in pretty good shape. Ideally we'd have one common code base but it may be that some configuration of components is done at build time, perhaps driven by 3 types, mobile, single instance, and clustered Does it make sense to anyone to think of this in the opposite direction, .i.e. upgrade/enhance BigCouch to the latest couchdb and then call that couchdb 2.0? Any other work from mobile builds and the like might be interesting to support. Were there any interesting changes to pull in from the mobile and embedded device ports? Also, I'm not sure where the view engine refactoring work ended up… I'll look at JIRA but maybe there are specific follow-ups to be made here. Finally, for our included javascript libraries and Futon runtime, are we going to replace or update anything here? I'd imagine a newer jQuery could be an advantage for those doing CouchApps. Brian. On Tuesday, February 14, 2012 at 7:28 , Noah Slater wrote: So far we have: https://issues.apache.org/jira/browse/COUCHDB-1410
Re: Roadmap for the 1.3.0 release
On Wed, Feb 15, 2012 at 2:13 PM, Bob Dionne dio...@dionne-associates.com wrote: On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote: Has there been any discussion around BigCouch integration strategies? It seems like it would fit the bill for the next undertaking on the general couch side. Does anyone from Cloudant have a suggestion for the timeline here? There's been a lot of discussion in #couchdb and #couchdb-dev but little on the ml. I'm not sure about timeline. There seems to be a lot of issues, most of them minor technical ones (the type that readily bog down once more than 3 people get involved). BigCouch embeds couchdb and was architected to be the clustering layer that couchdb lacks, so in that sense I think we're in pretty good shape. Ideally we'd have one common code base but it may be that some configuration of components is done at build time, perhaps driven by 3 types, mobile, single instance, and clustered Does it make sense to anyone to think of this in the opposite direction, .i.e. upgrade/enhance BigCouch to the latest couchdb and then call that couchdb 2.0? I was thinking that having a single instance that you could upgrade as a cluster member with just a configuration swicth would be a better plan. With smart rebalancing I could create cluster really dynamically. I understand that currently it will be hard technically to do that since couch embedded in bigcouch has been modified to get some infos from the cluster (like the design doc, validate func ...) but it's still possible. Not sure if it should happen first, but I really wish we follow this way rather than creating different instances types. - benoît
Re: Roadmap for the 1.3.0 release
On Wed, Feb 15, 2012 at 11:48 AM, Jan Lehnardt j...@apache.org wrote: On Feb 15, 2012, at 03:22 , Brian Mitchell wrote: Any other work from mobile builds and the like might be interesting to support. Were there any interesting changes to pull in from the mobile and embedded device ports? Also, I'm not sure where the view engine refactoring work ended up… I'll look at JIRA but maybe there are specific follow-ups to be made here. The biggest one IIRC was the use of emonk instead of an external Spidermonkey for iOS. Maybe we can make that a compile option like ./configure --enable-emonk-view-server so people can choose which one to run. There is some work started about that in the refuge project using old work done by @davisp used in couchbase and my emonk_helper. I will provide patches asap. Finally, for our included javascript libraries and Futon runtime, are we going to replace or update anything here? I'd imagine a newer jQuery could be an advantage for those doing CouchApps. Good that you bring this up. There's also other upstream dependencies that we should look at updating and we don't have a process for that, it usually happens whenever a developer feels like it. I haven't thought about this much yet, but maybe update all dependencies could be a todo before a release branch is created? +1 for a ticket :) Also include support for mobile support in futon would be fine. (but i think we should only depends on jquery rather than including another framework) . - benoit
Re: Roadmap for the 1.3.0 release
That sounds really neat, a number of folks have asked for such a thing. Right, the ddocs, validation funs, etc.. currently aren't stored globally, which requires clustered calls to retrieve them On Feb 15, 2012, at 8:21 AM, Benoit Chesneau wrote: On Wed, Feb 15, 2012 at 2:13 PM, Bob Dionne dio...@dionne-associates.com wrote: On Feb 14, 2012, at 9:22 PM, Brian Mitchell wrote: Has there been any discussion around BigCouch integration strategies? It seems like it would fit the bill for the next undertaking on the general couch side. Does anyone from Cloudant have a suggestion for the timeline here? There's been a lot of discussion in #couchdb and #couchdb-dev but little on the ml. I'm not sure about timeline. There seems to be a lot of issues, most of them minor technical ones (the type that readily bog down once more than 3 people get involved). BigCouch embeds couchdb and was architected to be the clustering layer that couchdb lacks, so in that sense I think we're in pretty good shape. Ideally we'd have one common code base but it may be that some configuration of components is done at build time, perhaps driven by 3 types, mobile, single instance, and clustered Does it make sense to anyone to think of this in the opposite direction, .i.e. upgrade/enhance BigCouch to the latest couchdb and then call that couchdb 2.0? I was thinking that having a single instance that you could upgrade as a cluster member with just a configuration swicth would be a better plan. With smart rebalancing I could create cluster really dynamically. I understand that currently it will be hard technically to do that since couch embedded in bigcouch has been modified to get some infos from the cluster (like the design doc, validate func ...) but it's still possible. Not sure if it should happen first, but I really wish we follow this way rather than creating different instances types. - benoît
[jira] [Commented] (COUCHDB-1398) improve view filtering in changes
[
https://issues.apache.org/jira/browse/COUCHDB-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208465#comment-13208465
]
Filipe Manana commented on COUCHDB-1398:
Benôit,
for 1) - Sure, no probem.
for 2) - That's right. It will work when the ddoc signature changes - my
comment there was if continuing to send changes after that happens is ok or not
for applications (since the view definition might have changed radically) -
it's probably ok for many applications. I don't have strong opinion here. If no
one objects it can remain like that imho.
Now for the the case the ddoc is deleted (and never recreated again), the
client will hang forever. The process spawned with spawn/1 will terminate with
a reason different from 'normal' (with a thow {not_found, deleted}) but the
parent will never know that, because it's not linked to it or monitoring it, so
it will hang forever waiting for db_updated or view_updated events (the later
won't come anymore). Like for normal continuous changes feeds we stop if the
database is deleted, for _view changes we should stop if the ddoc is deleted
(or its views attribute is deleted in an update) - this case seems to make
sense.
for 3) - I'm ok with that.
for 4) - Non manual tests are good to have :)
I've also noticed after last comment that send_view_changes/2 in
couch_changes.erl
(https://github.com/benoitc/couchdb/compare/master...couch_view_changes#L4R372)
is buffering the full_doc_infos of every document id found in the view when
folding it. This seems it can buffer millions (or more) of full_doc_info
records, no? If it's really unbounded (as it seems just by looking at the
diff), than it's dangerous (even a few thousand full_doc_infos can be too much,
if the revision trees have a big depth and/or many branches).,
improve view filtering in changes
-
Key: COUCHDB-1398
URL: https://issues.apache.org/jira/browse/COUCHDB-1398
Project: CouchDB
Issue Type: Improvement
Components: View Server Support
Affects Versions: 2.0, 1.3
Reporter: Benoit Chesneau
Labels: changes, view
Attachments: 0001-white-spaces.patch,
0002-initial-step-move-the-code-from-couch_httpd_db-to-co.patch,
0003-fix-indent.patch,
0004-This-wrapper-is-useless-somehow-split-the-code-in-a-.patch,
0005-add-view_updated-event.patch, 0006-immprove-view-filter.patch,
0007-useless-info.patch, 0008-whitespaces.patch,
0009-handle-native-filters-in-views.patch
Improve the native view filter `_view` support by really using view index.
This patches add following features
- small refactoring: create the couch_httpd_changes modules, to put all the
changes http support in its own module instead having it in couch_httpd_db.
- add the `view_updated` event when a view index is updated : {view_updated,
{DbName, IndexName}}
- start the feed using results in the view index instead of all the db index
- only react on view index changes.
For now next changes are still get using the couch_db:changes_since function
and passing the map function to the results. It could be improved if we had a
by_seq btree in the view index too. Other way may be to skip a number of the
documents already processed. Not sure it would be faster. Thoughts ?
The branch couch_view_changes in my repo contains preliminary support:
https://github.com/benoitc/couchdb/tree/couch_view_changes
Diff:
https://github.com/benoitc/couchdb/compare/master...couch_view_changes
To use it, use the native filter named _view which take the parameter
view=DesignName/Viewname
eg:
http://server/db/_changes?feed=continuousheartbeat=truefilter=_viewview=DesignName/SomeView
It has also an interresting side effect: on each db updates the view index
refresh is triggered so view updates are triggered. Maybe we could introduce
an optionnal parameter to not trigger them though?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1410) Formally define number support
[ https://issues.apache.org/jira/browse/COUCHDB-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208478#comment-13208478 ] Robert Newson commented on COUCHDB-1410: I appreciate that the format is fully defined. Perhaps what I mean, instead, is the precision with which those numbers can be manipulated in view servers? I've certainly been stung by some crazy number rounding issues in the past, I don't think it's reasonable behavior for a database. It sounds like this ticket is really two issues, 1) numbers can roundtrip safely to and from JSON, 2) numbers can be computed with within known (and consistent) bounds. Issue 1 is something we need to resolve in ejson for the 1.2.0 release but sounds simple. To fulfill this ticket, we have to commit to not breaking roundtrip safety in future versions. Issue 2, I suspect, is contentious. Or, at least, I suspect I desire stronger numeric handling than javascript typically delivers. I'll be happy here if we document, and preserve, some minimal standard. /ramble Formally define number support -- Key: COUCHDB-1410 URL: https://issues.apache.org/jira/browse/COUCHDB-1410 Project: CouchDB Issue Type: Improvement Affects Versions: 1.2 Reporter: Robert Newson Priority: Blocker Fix For: 1.3 The JSON spec has a very loose definition of Number. CouchDB, as a database, should have well-defined and first class support for numbers (both integral and decimal). The precision of number support should be formally specified as should the algorithm used to represent floating-point values, especially where an approximation must be made in the conversion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking the 1.2.0 release
On Wed, Feb 15, 2012 at 5:11 PM, Jan Lehnardt j...@apache.org wrote: Thus to have profiles, the admin must: 1. Add a .public = true key to the user doc. 2. Edit .views.lib.profiles_config and set enabled = true 3. Edit .views.lib.profiles_config and add some fields to the whitelist (e.g. [gravatar_url]) Do you want to put this into a JIRA for 1.2.1 and 1.3.0? Love it. Does that imply that views from system databases are a possibility? I thought they were a no-go, or is that just for 1.2.0? Thanks. -- Iris Couch
[jira] [Commented] (COUCHDB-1410) Formally define number support
[ https://issues.apache.org/jira/browse/COUCHDB-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208491#comment-13208491 ] Jason Smith commented on COUCHDB-1410: -- Is Javascript relevant? The storage layer reads and writes documents. Everything arrives or departs (whether to HTTP layer or to a view server) as JSON. What couchjs does with that JSON seems a different issue. What my browser does with the JSON is a different issue. I like Paul's point to simply document how it works and leave it at that. It's basically how most databases work. Maybe Couch 3000 would improve on that, but for today... Formally define number support -- Key: COUCHDB-1410 URL: https://issues.apache.org/jira/browse/COUCHDB-1410 Project: CouchDB Issue Type: Improvement Affects Versions: 1.2 Reporter: Robert Newson Priority: Blocker Fix For: 1.3 The JSON spec has a very loose definition of Number. CouchDB, as a database, should have well-defined and first class support for numbers (both integral and decimal). The precision of number support should be formally specified as should the algorithm used to represent floating-point values, especially where an approximation must be made in the conversion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking the 1.2.0 release
On Feb 15, 2012, at 14:55 , Jason Smith wrote: On Wed, Feb 15, 2012 at 5:11 PM, Jan Lehnardt j...@apache.org wrote: Thus to have profiles, the admin must: 1. Add a .public = true key to the user doc. 2. Edit .views.lib.profiles_config and set enabled = true 3. Edit .views.lib.profiles_config and add some fields to the whitelist (e.g. [gravatar_url]) Do you want to put this into a JIRA for 1.2.1 and 1.3.0? Love it. Does that imply that views from system databases are a possibility? I thought they were a no-go, or is that just for 1.2.0? Admin users can create and query as many views as they want. Cheers Jan --
Re: Issues blocking the 1.2.0 release
On Wednesday, February 15, 2012, Jason Smith j...@iriscouch.com wrote: On Wed, Feb 15, 2012 at 5:11 PM, Jan Lehnardt j...@apache.org wrote: Thus to have profiles, the admin must: 1. Add a .public = true key to the user doc. 2. Edit .views.lib.profiles_config and set enabled = true 3. Edit .views.lib.profiles_config and add some fields to the whitelist (e.g. [gravatar_url]) Do you want to put this into a JIRA for 1.2.1 and 1.3.0? Love it. Does that imply that views from system databases are a possibility? I thought they were a no-go, or is that just for 1.2.0? Thanks. -- Iris Couch imo profiles should go in a profile db. admin views can already be created by admins. benoît
