Re: [jira] Resolved: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
On Jul 30, 2009, at 1:55 PM, Paul Davis wrote: ... Which upcoming patches? Are these developments in progress or somewhere down the road. Have they been discussed on the mailing list (dates or topics would be useful)? Are there JIRA entries for them? I was specifically thinking of the OAuth stuff that Jason Davies has been working on lately. There's also the cookie based auth. I'm not sure if anyone ever took up the mantle for ldap integration or not. But in general, there has been movement on authentication mechanisms lately. It is good to avoid the case where a feature arrives fully baked in the SVN without previous discussion. It gives the community has no opportunity to see if it meets their needs before it is cast in code. It sure is. Jason has put his patches for OAuth up on JIRA [1]. I think Benoit's cookie auth has a ticket in there somewhere. Had those on my radar with good intentions to review them this weekend. Something about the previous message just triggered the thought that there might have been something else brewing. I've been trying to organize info from the mailing list, JIRA entries and code review into an overview of Authentication and Authorization thoughts on the Wiki It hasn't been easy getting up to speed and I'd to make things easier for anyone else who is trying to catch up. We greatly appreciate your help on focusing the picture on the auth/access work. If you're getting stumped with Google and the ML archives, your best bet is to hop on IRC and ask your questions. Its a friendly channel and there are starting to be enough regulars in all timezones that someone's always around. HTH, Paul Davis [1] https://issues.apache.org/jira/browse/COUCHDB-420 Thanks.
[jira] Updated: (COUCHDB-406) build on windows
[ https://issues.apache.org/jira/browse/COUCHDB-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hammond updated COUCHDB-406: - Attachment: windows_build_2.patch updated patch addressing comments from nslater, and updated README based on starting this process again from scratch. I'm proposing this for checkin ASAP. > build on windows > > > Key: COUCHDB-406 > URL: https://issues.apache.org/jira/browse/COUCHDB-406 > Project: CouchDB > Issue Type: Improvement > Components: Build System > Environment: windows, trunk >Reporter: Mark Hammond > Attachments: windows_build.patch, windows_build_2.patch > > > Attaching a patch to get things building on Windows, including docs. As > noted in the docs, we reuse as much of the couch and erlang build processes > as possible. This patch leaves a functioning couch on Windows - with some > polish left to be done, but working and tests passing as well as expected - > and still works on linux. > Includes a couple of new files - couchspawnkillable_win.c from COUCHDB-374 > (while I will close), and a couchdb.bat.tpl file which generates a suitable > .bat file at 'make install' time. > Note this patch makes no attempt to create an installer - it just creates a > functioning tree suitable for zipping up, or for future consumption by an > installer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
[ https://issues.apache.org/jira/browse/COUCHDB-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737411#action_12737411 ] Adam Kocoloski commented on COUCHDB-442: Hmm Chris, I'm not sure I agree with that statement. I've seen production setups with ~5 GB view indexes where the view emitted (key,null) and used include_docs=true for the lookup. If the view had used sanitize(doc) it would've been much too big to be cached in main memory. Then you're pretty well screwed. > Add a "view" or "format" function to process source doc on query > > > Key: COUCHDB-442 > URL: https://issues.apache.org/jira/browse/COUCHDB-442 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Reporter: Curt Arnold > > It is common practice to do emit(key, null) in a map function and then query > with ?include_docs=true to retrieve the documents that were responsible for > the entries. However, the full document may include information that is > privileged or the full document may be substantially larger than the > information needed to be transferred to the client. > The proposed enhancement is to allow defining a "view" function in addition > to the existing "map" and "reduce" on a view. If specified, the view > function would take the id, key, value and doc and return a JSON value that > would be added as the "view" member to the row in the result set. > One of the use cases on > http://wiki.apache.org/couchdb/Authentication_and_Authorization is to be able > to specify that a user can retrieve the values from a view, but not add > include_docs since that may expose information that they are not authorized > to view. Without the "view" function, there would be pressure to start > pushing things into the emitted value. > Production of views would be likely controlled using a include_views=true in > the query string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
[ https://issues.apache.org/jira/browse/COUCHDB-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737340#action_12737340 ] Chris Anderson commented on COUCHDB-442: for what it's worth: emit(key, sanitize(doc)); this is the "right" way to do it. include_docs should be avoided in production code paths (admin is fine) especially where you'll be selecting a non-trivial number of rows. > Add a "view" or "format" function to process source doc on query > > > Key: COUCHDB-442 > URL: https://issues.apache.org/jira/browse/COUCHDB-442 > Project: CouchDB > Issue Type: Improvement > Components: JavaScript View Server >Reporter: Curt Arnold > > It is common practice to do emit(key, null) in a map function and then query > with ?include_docs=true to retrieve the documents that were responsible for > the entries. However, the full document may include information that is > privileged or the full document may be substantially larger than the > information needed to be transferred to the client. > The proposed enhancement is to allow defining a "view" function in addition > to the existing "map" and "reduce" on a view. If specified, the view > function would take the id, key, value and doc and return a JSON value that > would be added as the "view" member to the row in the result set. > One of the use cases on > http://wiki.apache.org/couchdb/Authentication_and_Authorization is to be able > to specify that a user can retrieve the values from a view, but not add > include_docs since that may expose information that they are not authorized > to view. Without the "view" function, there would be pressure to start > pushing things into the emitted value. > Production of views would be likely controlled using a include_views=true in > the query string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: sending head of a view result in one chunk
On Wed, Jul 29, 2009 at 08:30:33PM +0200, Benoit Chesneau wrote:
> Following discussion on irc I post here my problem. I am currently
> implementing a feature allowing me to parse the view while it's coming
> over http. Using chunks allow me to parse each line easily. It
> currently work but currently first line of view is sent with the
> header. It would be better imo to send the header and the first line
> in different chunks, so It isn't needed to split the result. What do
> you think about it ?
Are you talking about HTTP chunking? I think it's an extremely bad idea to
rely on HTTP chunk boundaries to have significance in the sub-parts of the
body. Maybe CouchDB could generate chunks like that, but it could all be
lost by any intermediate proxy.
For parsing views in a streaming manner, CouchDB inserts a newline after
each row. This is a frig, but at least it's a portable frig. CouchRest
handles this quite happily. The headers and body are separated by \r\n\r\n
of course, and there's not much you can do about that.
Or do you mean sending a newline after the opening part of the JSON doc?
This is already done, as far as I can see.
$ curl http://127.0.0.1:5984/testdb/_all_docs
{"total_rows":1,"offset":0,"rows":[
{"id":"12345","key":"12345","value":{"rev":"1-171352454"}}
]}
I may have misunderstood you completely though.
Regards,
Brian.
Re: [jira] Resolved: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
On Thu, Jul 30, 2009 at 9:11 AM, Curt Arnold wrote: > > On Jul 29, 2009, at 11:02 PM, Paul Joseph Davis (JIRA) wrote: > >> >> [ >> https://issues.apache.org/jira/browse/COUCHDB-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] >> >> Paul Joseph Davis resolved COUCHDB-442. >> --- >> >> Resolution: Later >> >> Right, we'd need to add ACL support to _list and _show to accomplish the >> original intent, but that was my reason for doubting the feature; we can >> just roll this functionality into the existing feature set. >> >> Closing this because it probably won't happen. When some of the upcoming >> auth/access patches hit similar functionality will most likely get rolled >> into _list and _show. For interested listeners, keep an ear to the dev@ >> ground for updates. > > > Which upcoming patches? Are these developments in progress or somewhere > down the road. Have they been discussed on the mailing list (dates or > topics would be useful)? Are there JIRA entries for them? > I was specifically thinking of the OAuth stuff that Jason Davies has been working on lately. There's also the cookie based auth. I'm not sure if anyone ever took up the mantle for ldap integration or not. But in general, there has been movement on authentication mechanisms lately. > It is good to avoid the case where a feature arrives fully baked in the SVN > without previous discussion. It gives the community has no opportunity to > see if it meets their needs before it is cast in code. > It sure is. Jason has put his patches for OAuth up on JIRA [1]. I think Benoit's cookie auth has a ticket in there somewhere. > I've been trying to organize info from the mailing list, JIRA entries and > code review into an overview of Authentication and Authorization thoughts on > the Wiki It hasn't been easy getting up to speed and I'd to make things > easier for anyone else who is trying to catch up. > We greatly appreciate your help on focusing the picture on the auth/access work. If you're getting stumped with Google and the ML archives, your best bet is to hop on IRC and ask your questions. Its a friendly channel and there are starting to be enough regulars in all timezones that someone's always around. HTH, Paul Davis [1] https://issues.apache.org/jira/browse/COUCHDB-420
[jira] Commented: (COUCHDB-441) Finally implement pre-write-doc-edit handlers.
[ https://issues.apache.org/jira/browse/COUCHDB-441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737214#action_12737214 ] Curt Arnold commented on COUCHDB-441: - ?include_doc=true could be borrowed from the view syntax so that you got the committed doc back as the "doc" member of the reply regardless of whether there was any pre-write-doc-edit handler. If you aren't modifying or you don't need the doc, you could omit the param or set it to false. I'd discourage special replacements sequences, then you have to figure out how to escape them when you really wanted a literal "$now$" in the body. > Finally implement pre-write-doc-edit handlers. > -- > > Key: COUCHDB-441 > URL: https://issues.apache.org/jira/browse/COUCHDB-441 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Affects Versions: 0.10 >Reporter: Curt Arnold > Fix For: 0.10 > > > It would be useful for auditing to have the identity of the user who inserted > a new revision and the timestamp of the operation to be inserted in the > document in the same way that the new revision number is. > Doing this at the application level is not adequate since it would be readily > spoofable and would bypass the authentication handler. > There is a comment in couch_db:update_docs about generating new revision ids, > but I couldn't quite comprehend what specific code was responsible for > inserting the id into the document. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-441) Finally implement pre-write-doc-edit handlers.
[
https://issues.apache.org/jira/browse/COUCHDB-441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737187#action_12737187
]
Robert Burke commented on COUCHDB-441:
--
Being able to insert info in docs when committed to the db implies that using
PUT/POST to create/update a document needs to return more in the response than
what it currently does, for example: {"ok":true, "id":"123BAC",
"rev":"946B7D1C"}
It probably needs to return the entire JSON of the new or updated doc.
Otherwise, gotta call the db twice.
> Finally implement pre-write-doc-edit handlers.
> --
>
> Key: COUCHDB-441
> URL: https://issues.apache.org/jira/browse/COUCHDB-441
> Project: CouchDB
> Issue Type: Improvement
> Components: HTTP Interface
>Affects Versions: 0.10
>Reporter: Curt Arnold
> Fix For: 0.10
>
>
> It would be useful for auditing to have the identity of the user who inserted
> a new revision and the timestamp of the operation to be inserted in the
> document in the same way that the new revision number is.
> Doing this at the application level is not adequate since it would be readily
> spoofable and would bypass the authentication handler.
> There is a comment in couch_db:update_docs about generating new revision ids,
> but I couldn't quite comprehend what specific code was responsible for
> inserting the id into the document.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: Time to move bugspam to a separate list?
On Thu, Jul 30, 2009 at 06:01:41PM +0200, Volker Mische wrote: > Just another reminder: > I would send the MoinMoin changes to comm...@couchdb.apache.org instead > of couchdb-comm...@incubator.apache.org :) Who do we ping about this? -- Noah Slater, http://tumbolia.org/nslater
Re: Time to move bugspam to a separate list?
Noah Slater wrote: > On Thu, Jul 30, 2009 at 10:27:41AM +0200, Dirkjan Ochtman wrote: >> Now that volume for both conversation and usage of the bugtracker has >> increased, might this be a good time to move all JIRA mail to a >> different list? >> >> I like to track high-level discussions, but most of the bugs aren't >> that interesting to me. It'd be nice to bring volume down a little. Of >> course I could filter them out client-side, but I thought I'd float >> this here first. > > I'm not sure what is common with the ASF, but we have three things: > > * Commit messages from Subversion > > * Change messages from MoinMoin > > * Change messages from JIRA > > Developers and committers should be monitoring them all, really. > > hey all need to go some place, but where? > > We currently have: > > u...@couchdb.apache.org > > dev@couchdb.apache.org > > comm...@couchdb.apache.org > > How would you distribute these messages? > > Would you create a new list? > > Best, > Just another reminder: I would send the MoinMoin changes to comm...@couchdb.apache.org instead of couchdb-comm...@incubator.apache.org :) Cheers, Volker
Re: translating the couchdb site into Japanese
On Thu, Jul 30, 2009 at 04:40:41PM +0900, Hideo Yanagi wrote: > My small team and I are starting to translate the contents at > couchdb.apache.org. We see a great value and potential in the CouchDB > technologies, and we would like to contribute to the project by > expanding user base and community in Japan. Fantastic. However, I think there may be some issues around this. Does the ASF have any policy on how we handle translations? Are translations done under the auspices of the main project? Is there any documented policy? Mentors? How is the wiki licensed? Did we progress with that? Thanks, -- Noah Slater, http://tumbolia.org/nslater
Re: Time to move bugspam to a separate list?
On Thu, Jul 30, 2009 at 10:27:41AM +0200, Dirkjan Ochtman wrote: > Now that volume for both conversation and usage of the bugtracker has > increased, might this be a good time to move all JIRA mail to a > different list? > > I like to track high-level discussions, but most of the bugs aren't > that interesting to me. It'd be nice to bring volume down a little. Of > course I could filter them out client-side, but I thought I'd float > this here first. I'm not sure what is common with the ASF, but we have three things: * Commit messages from Subversion * Change messages from MoinMoin * Change messages from JIRA Developers and committers should be monitoring them all, really. hey all need to go some place, but where? We currently have: u...@couchdb.apache.org dev@couchdb.apache.org comm...@couchdb.apache.org How would you distribute these messages? Would you create a new list? Best, -- Noah Slater, http://tumbolia.org/nslater
[jira] Commented: (COUCHDB-441) Finally implement pre-write-doc-edit handlers.
[ https://issues.apache.org/jira/browse/COUCHDB-441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737136#action_12737136 ] Robert Burke commented on COUCHDB-441: -- We could use either a generic hook to insert timestamps, or some built-in handling where possibly the string $now$ in the JSON (possibly with a format specifier to get ISO 8601 dates or other date formats) gets replaced with the current UTC time. We have many timestamps that we need to record other than just created / last modified (when the user completed a certain workflow step, etc). We can keep a few db machines in close enough synch with a time server. We cannot rely on our client machines to have the correct time. > Finally implement pre-write-doc-edit handlers. > -- > > Key: COUCHDB-441 > URL: https://issues.apache.org/jira/browse/COUCHDB-441 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Affects Versions: 0.10 >Reporter: Curt Arnold > Fix For: 0.10 > > > It would be useful for auditing to have the identity of the user who inserted > a new revision and the timestamp of the operation to be inserted in the > document in the same way that the new revision number is. > Doing this at the application level is not adequate since it would be readily > spoofable and would bypass the authentication handler. > There is a comment in couch_db:update_docs about generating new revision ids, > but I couldn't quite comprehend what specific code was responsible for > inserting the id into the document. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: [jira] Resolved: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
On 30 Jul 2009, at 15:11, Curt Arnold wrote: On Jul 29, 2009, at 11:02 PM, Paul Joseph Davis (JIRA) wrote: [ https://issues.apache.org/jira/browse/COUCHDB-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Joseph Davis resolved COUCHDB-442. --- Resolution: Later Right, we'd need to add ACL support to _list and _show to accomplish the original intent, but that was my reason for doubting the feature; we can just roll this functionality into the existing feature set. Closing this because it probably won't happen. When some of the upcoming auth/access patches hit similar functionality will most likely get rolled into _list and _show. For interested listeners, keep an ear to the dev@ ground for updates. Which upcoming patches? Are these developments in progress or somewhere down the road. Have they been discussed on the mailing list (dates or topics would be useful)? Are there JIRA entries for them? It is good to avoid the case where a feature arrives fully baked in the SVN without previous discussion. It gives the community has no opportunity to see if it meets their needs before it is cast in code. I've been trying to organize info from the mailing list, JIRA entries and code review into an overview of Authentication and Authorization thoughts on the Wiki It hasn't been easy getting up to speed and I'd to make things easier for anyone else who is trying to catch up. https://issues.apache.org/jira/browse/COUCHDB-441 :) Cheers Jan --
[jira] Commented: (COUCHDB-263) require valid user for all database operations
[ https://issues.apache.org/jira/browse/COUCHDB-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737097#action_12737097 ] Jason Davies commented on COUCHDB-263: -- I've absorbed this patch into my oauth branch at http://github.com/jasondavies/couchdb/tree/oauth . I've modified it as follows: 1. The setting has been moved to [couch_httpd_auth] require_valid_user = true 2. The setting affects all authentication handlers instance-wide. If none of them set user_ctx, then a 401 error is returned when require_valid_user = true. > require valid user for all database operations > -- > > Key: COUCHDB-263 > URL: https://issues.apache.org/jira/browse/COUCHDB-263 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Affects Versions: 0.9 > Environment: All platforms. >Reporter: Jack Moffitt >Priority: Blocker > Fix For: 0.10 > > Attachments: couchauth.diff > > > Admin accounts currently restrict a few operations, but leave all other > operations completely open. Many use cases will require all operations to be > authenticated. This can certainly be done by overriding the > default_authentication_handler, but I think this very common use case can be > handled in default_authentication_handler without increasing the complexity > much. > Attached is a patch which adds a new config option, "require_valid_user", > which restricts all operations to authenticated users only. Since CouchDB > currently only has admins, this means that all operations are restricted to > admins. In a future CouchDB where there are also normal users, the intention > is that this would let them pass through as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: [jira] Resolved: (COUCHDB-442) Add a "view" or "format" function to process source doc on query
On Jul 29, 2009, at 11:02 PM, Paul Joseph Davis (JIRA) wrote: [ https://issues.apache.org/jira/browse/COUCHDB-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Joseph Davis resolved COUCHDB-442. --- Resolution: Later Right, we'd need to add ACL support to _list and _show to accomplish the original intent, but that was my reason for doubting the feature; we can just roll this functionality into the existing feature set. Closing this because it probably won't happen. When some of the upcoming auth/access patches hit similar functionality will most likely get rolled into _list and _show. For interested listeners, keep an ear to the dev@ ground for updates. Which upcoming patches? Are these developments in progress or somewhere down the road. Have they been discussed on the mailing list (dates or topics would be useful)? Are there JIRA entries for them? It is good to avoid the case where a feature arrives fully baked in the SVN without previous discussion. It gives the community has no opportunity to see if it meets their needs before it is cast in code. I've been trying to organize info from the mailing list, JIRA entries and code review into an overview of Authentication and Authorization thoughts on the Wiki It hasn't been easy getting up to speed and I'd to make things easier for anyone else who is trying to catch up.
Time to move bugspam to a separate list?
Now that volume for both conversation and usage of the bugtracker has increased, might this be a good time to move all JIRA mail to a different list? I like to track high-level discussions, but most of the bugs aren't that interesting to me. It'd be nice to bring volume down a little. Of course I could filter them out client-side, but I thought I'd float this here first. Cheers, Dirkjan
translating the couchdb site into Japanese
Dear CouchDB developers, My name is Hideo Yanagi, and I am an engineer and tranlsator from Tokyo, Japan. My small team and I are starting to translate the contents at couchdb.apache.org. We see a great value and potential in the CouchDB technologies, and we would like to contribute to the project by expanding user base and community in Japan. We are thinking of starting from the static contents at the main site, and then move on to the wiki contents later. If yo have tips or suggestions for the translation, especially on the dynamic wiki contents, please let me know. Any suggestions will be greatly appreciated. Also, if there is any copyright related issue for the translation, please kindly let me know. Best Regards, Hideo Yanagi hideo.yan...@gmail.com
