updated instructions for windows from-scratch build
Hi folks A lot longer than expected getting this out the door ran into more issues trying to run in a virtual environment (beer sob stories later). I'd appreciate a look over any comments back, while I split this longer doc back out into updates for INSTALL.Windows then patch. I've already got the upstream Erlang changes ready to go now. http://github.com/dch/glazier/#readme Goal of glazier is to build a fully automated windows + erlang + couch compile chain. Maybe use in blame testing - do we have a windows apache build infrastructure that could be used? The same manual steps in glazier should make it much more clear in future for non-automated builds also. Thanks Dave
[jira] Commented: (COUCHDB-886) Add option to set query options, defined in rewrites.json, as default
[
https://issues.apache.org/jira/browse/COUCHDB-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908721#action_12908721
]
Benoit Chesneau commented on COUCHDB-886:
-
the problem with all of this is to allows the users to override query
parameters.
Ex if I have this rule:
{
from: /city/paris,
to: _list/test,
query: {
startkey: paris,
'endkey: :paris\0
}
If we make query overridable, that mean that someone passing starkey=endkey=
will override the path rule. Here we are in a simple case so it doesn't really
matter, but it could be worth. If we have default on the other hand, for
example with a limit parameter :
{
from: /city/paris,
to: _list/test,
query: {
startkey: paris,
endkey: :paris\0,
limit: :llimit
},
default: {
limit: 10
}
}
Then if we do /city/paris, the default limit will be use while if we do
/city/paris?limit=50, limit query parameter will be used.
Imo, default: {} is really the tight way to handle such problem.
Add option to set query options, defined in rewrites.json, as default
-
Key: COUCHDB-886
URL: https://issues.apache.org/jira/browse/COUCHDB-886
Project: CouchDB
Issue Type: Bug
Components: Database Core
Reporter: Henrik Skupin
With the latest version of CouchDB the URL parameters are not taken into
account when the rewrites.json file specifies the same ones for the
appropriate entry. See the following example:
{
from : /general/reports,
to : _list/general_reports/general_reportsByDate,
query : {
descending : true,
limit : 51
}
}
default values: http://mozmill.hskupin.info/general/reports
custom values: http://mozmill.hskupin.info/general/reports?limit=10
Whether which URL you are loading, the values from rewrites.json are always
used. Once the limit entry gets removed from that file, the URL parameter is
used and 10 rows are displayed.
As proposed by Benoit query entries should be explicitly allowed to have a
default value. Otherwise the value from rewrites.json has the priority.
query: {
key: { value: :var, default: 1}
}
Can true be used instead of a number? It could be confusing this way,
especially when the value is also a number.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-884) Improve error handling in couch_rep_reader
[
https://issues.apache.org/jira/browse/COUCHDB-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Filipe Manana closed COUCHDB-884.
-
Resolution: Fixed
Applied to trunk (revision 996492) and 1.0.x (revision 996493)
Improve error handling in couch_rep_reader
--
Key: COUCHDB-884
URL: https://issues.apache.org/jira/browse/COUCHDB-884
Project: CouchDB
Issue Type: Improvement
Components: Replication
Affects Versions: 1.0.1
Reporter: Filipe Manana
Assignee: Filipe Manana
Priority: Minor
Attachments: rep_reader_errors.patch
Currently, the replicator might crash when trying to read a doc from a remote
DB. This is due unmatched clauses in a function. Example:
** Last message in was {'DOWN',#Ref0.0.0.687,process,0.216.0,
{function_clause,
[{couch_rep_reader,'-open_doc_revs/3-fun-1-',
[{[{error,unauthorized},
{reason,Authentication required.}]},
The following patch avoids that the replicator crashes in this case and adds
a useful error log message as well.
Is anyone against applying this to trunk and 1.0.x? Adam?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-882) Nonstandard HTTP methods not converted to JSON correctly
[ https://issues.apache.org/jira/browse/COUCHDB-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Smith updated COUCHDB-882: Attachment: 0001-For-unknown-HTTP-methods-Use-a-binary-for-httpd.meth.patch Simplified patch which simply converts any non-atom received from mochi into binary. Nonstandard HTTP methods not converted to JSON correctly Key: COUCHDB-882 URL: https://issues.apache.org/jira/browse/COUCHDB-882 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 1.0.1 Environment: Erlang R13, Linux Reporter: Jason Smith Priority: Minor Fix For: 1.0.1 Attachments: 0001-For-unknown-HTTP-methods-Use-a-binary-for-httpd.meth.patch, 0001-For-unknown-HTTP-methods-Use-a-binary-for-httpd.meth.patch Since COUCHDB-815, CouchDB allows nonstandard or unknown HTTP methods in case a _show or similar function may want to implement a response to that method. Unfortunately the (my) patch in that ticket used couch_util:to_existing_atom which returns the passed value unmodified if it has no corresponding atom. That is wrong because the HTTP method will be copied into the `req` object in the view server, therefore it must not be an Erlang string (list of integers) because those do not JSONify correctly. Instead, if the atom does not exist, the method should be converted to a binary. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-886) Add option to set query options, defined in rewrites.json, as default
[
https://issues.apache.org/jira/browse/COUCHDB-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908927#action_12908927
]
Ryan Hill commented on COUCHDB-886:
---
Why not split the difference?
Let 'query' be strictly immutable, and let URL parameters override 'default'
values.
Values in 'query' would also override 'default' values.
Seems like a relaxed approach to me.
Add option to set query options, defined in rewrites.json, as default
-
Key: COUCHDB-886
URL: https://issues.apache.org/jira/browse/COUCHDB-886
Project: CouchDB
Issue Type: Bug
Components: Database Core
Reporter: Henrik Skupin
With the latest version of CouchDB the URL parameters are not taken into
account when the rewrites.json file specifies the same ones for the
appropriate entry. See the following example:
{
from : /general/reports,
to : _list/general_reports/general_reportsByDate,
query : {
descending : true,
limit : 51
}
}
default values: http://mozmill.hskupin.info/general/reports
custom values: http://mozmill.hskupin.info/general/reports?limit=10
Whether which URL you are loading, the values from rewrites.json are always
used. Once the limit entry gets removed from that file, the URL parameter is
used and 10 rows are displayed.
As proposed by Benoit query entries should be explicitly allowed to have a
default value. Otherwise the value from rewrites.json has the priority.
query: {
key: { value: :var, default: 1}
}
Can true be used instead of a number? It could be confusing this way,
especially when the value is also a number.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-886) Add option to set query options, defined in rewrites.json, as default
[
https://issues.apache.org/jira/browse/COUCHDB-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908938#action_12908938
]
Chris Anderson commented on COUCHDB-886:
Ryan's proposal is what I was proposing:
the required-query stuff overrides any defaults (if both specify a param)
I'm not sure about naming, I just want it to be clear to users which is which...
so defaults and query (maybe 'defaults' is better than 'default' as
'default' is a javascript keyword, which can be a surprise if you use it
unquoted)
or
query and enforced-query
or something else.
The only requirement being that one of them is named query for the sake of
backwards compatibility.
Add option to set query options, defined in rewrites.json, as default
-
Key: COUCHDB-886
URL: https://issues.apache.org/jira/browse/COUCHDB-886
Project: CouchDB
Issue Type: Bug
Components: Database Core
Reporter: Henrik Skupin
With the latest version of CouchDB the URL parameters are not taken into
account when the rewrites.json file specifies the same ones for the
appropriate entry. See the following example:
{
from : /general/reports,
to : _list/general_reports/general_reportsByDate,
query : {
descending : true,
limit : 51
}
}
default values: http://mozmill.hskupin.info/general/reports
custom values: http://mozmill.hskupin.info/general/reports?limit=10
Whether which URL you are loading, the values from rewrites.json are always
used. Once the limit entry gets removed from that file, the URL parameter is
used and 10 rows are displayed.
As proposed by Benoit query entries should be explicitly allowed to have a
default value. Otherwise the value from rewrites.json has the priority.
query: {
key: { value: :var, default: 1}
}
Can true be used instead of a number? It could be confusing this way,
especially when the value is also a number.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-886) Add option to set query options, defined in rewrites.json, as default
[
https://issues.apache.org/jira/browse/COUCHDB-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908950#action_12908950
]
Benoit Chesneau commented on COUCHDB-886:
-
so just t be clear process is :
query url query params query-overrides (or defaults whatever) - final
query params ?
- benoit
Add option to set query options, defined in rewrites.json, as default
-
Key: COUCHDB-886
URL: https://issues.apache.org/jira/browse/COUCHDB-886
Project: CouchDB
Issue Type: Bug
Components: Database Core
Reporter: Henrik Skupin
With the latest version of CouchDB the URL parameters are not taken into
account when the rewrites.json file specifies the same ones for the
appropriate entry. See the following example:
{
from : /general/reports,
to : _list/general_reports/general_reportsByDate,
query : {
descending : true,
limit : 51
}
}
default values: http://mozmill.hskupin.info/general/reports
custom values: http://mozmill.hskupin.info/general/reports?limit=10
Whether which URL you are loading, the values from rewrites.json are always
used. Once the limit entry gets removed from that file, the URL parameter is
used and 10 rows are displayed.
As proposed by Benoit query entries should be explicitly allowed to have a
default value. Otherwise the value from rewrites.json has the priority.
query: {
key: { value: :var, default: 1}
}
Can true be used instead of a number? It could be confusing this way,
especially when the value is also a number.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-885) Attachments introduce conflicts after replication.
[
https://issues.apache.org/jira/browse/COUCHDB-885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolai Teofilov updated COUCHDB-885:
-
Summary: Attachments introduce conflicts after replication. (was:
Delete document with attachment fails after replication.)
Description:
Step to reproduce the bug:
1. Make database test on a remote couchdb server that reside on a different
machine!
2. Create new document: http://remote-server:5984/test/doc;
3. Create database replica on the local couchdb server.
4. Trigger pull replication http://remote-server:5984/test -
http://localhost:5984/replica
5. Attach a file to the replicated document on the local couchdb server.
6. Trigger push replication http://localhost:5984/replica -
http://remote-server:5984/test
The document in the test database shows one conflict. If instead attachment
the document is updated with a new field after the replication no conflict.
The same sequence shows no conflicts on CouchDB 0.11.
test.sh
#!/usr/bin/env bash
LOCAL=http://localhost:5984;
REMOTE=http://koleto.couchone.com;
echo =
echo This sequence create a conflict after the replication
echo =
echo Cleanup
curl --silent -X DELETE $LOCAL/replica
curl --silent -X DELETE $REMOTE/test
echo Create database: $REMOTE/test
curl --silent -X PUT $REMOTE/test
echo Create empty document:
curl --silent -d '{}' -X PUT $REMOTE/test/doc
echo Create local databse: $LOCAL/replica
curl --silent -X PUT $LOCAL/replica
echo Pull replication: REMOTE/test - $LOCAL/replica
curl --silent -X POST -d
{\source\:\$REMOTE/test\,\target\:\$LOCAL/replica\} -H 'Content-Type:
application/json' $LOCAL/_replicate
echo Put an attachment to the document:
echo foobar! | curl --silent --upload-file - -X PUT
$LOCAL/replica/doc/foo.txt?rev=1-967a00dff5e02add41819138abb3284d
echo Push replication: $LOCAL/replica - $REMOTE/test
curl --silent -X POST -d
{\source\:\$LOCAL/replica\,\target\:\$REMOTE/test\} -H 'Content-Type:
application/json' $LOCAL/_replicate
echo '## Get one conflic #'
curl --silent -X GET $REMOTE/test/doc?conflicts=true
echo =
echo The same sequence with adding a field to the document
echo =
echo Cleanup
curl --silent -X DELETE $LOCAL/replica
curl --silent -X DELETE $REMOTE/test
echo Create database: $REMOTE/test
curl --silent -X PUT $REMOTE/test
echo Create empty document:
curl --silent -d '{}' -X PUT $REMOTE/test/doc
echo Create local databse: $LOCAL/replica
curl --silent -X PUT $LOCAL/replica
echo Pull replication: REMOTE/test - $LOCAL/replica
curl --silent -X POST -d
{\source\:\$REMOTE/test\,\target\:\$LOCAL/replica\} -H 'Content-Type:
application/json' $LOCAL/_replicate
echo Adding a new field:
curl --silent -d '{foo:bar , _rev : 1-967a00dff5e02add41819138abb3284d
}' -X PUT $LOCAL/replica/doc?rev=1-967a00dff5e02add41819138abb3284d
echo Push replication: $LOCAL/replica - $REMOTE/test
curl --silent -X POST -d
{\source\:\$LOCAL/replica\,\target\:\$REMOTE/test\} -H 'Content-Type:
application/json' $LOCAL/_replicate
echo '## No conflicts after the replication '
curl --silent -X GET $REMOTE/test/doc?conflicts=true
was:
Step to reproduce the bug:
1. Make database test on a remote couchdb server that reside on a different
machine!
2. Create new document: http://remote-server:5984/test/doc;
3. Create database test on the local couchdb server.
4. Trigger pull replication http://remote-server:5984/test -
http://localhost:5984/test
5. Attach a file to the replicated document on the local couchdb server.
6. Trigger push replication http://localhost:5984/test -
http://remote-server:5984/test
7. Delete the replicated document that contain now the attachment on remote
database.
This operation will delete the last revision of the document (after the
replication) but the previous revision of the document (before the replication)
still exist in the database.
This defect appears only for replications between databases on two different
couchdb servers, and only for documents that were updated with a new attachment.
Fix Version/s: 0.11
Component/s: Database Core
(was: Replication)
Attachments introduce conflicts after replication.
--
Key: COUCHDB-885
URL: https://issues.apache.org/jira/browse/COUCHDB-885
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.0.1
Environment: Mac OSX, Windows XP, Windows 7
Reporter: Nikolai Teofilov
Fix For: 0.11
Step to reproduce
[jira] Created: (COUCHDB-887) _log handler has very odd semantics for bytes/offset, probably a bug
_log handler has very odd semantics for bytes/offset, probably a bug Key: COUCHDB-887 URL: https://issues.apache.org/jira/browse/COUCHDB-887 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 1.0.1, 1.0, 0.11.2, 0.11.1, 0.11 Reporter: Nuutti Kotivuori Priority: Trivial The _log method for couchdb servers has query arguments bytes and offset, but they seem to behave really weirdly. If offset = bytes, an eof error is returned - otherwise offset is just substracted from bytes and the request behaves identically otherwise. A simple fix to the expected behaviour is included - offset counts backwards from bytes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-887) _log handler has very odd semantics for bytes/offset, probably a bug
[ https://issues.apache.org/jira/browse/COUCHDB-887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nuutti Kotivuori updated COUCHDB-887: - Attachment: couch_log.erl.diff Patch to fix behavior to be possibly as expected. _log handler has very odd semantics for bytes/offset, probably a bug Key: COUCHDB-887 URL: https://issues.apache.org/jira/browse/COUCHDB-887 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 0.11, 0.11.1, 0.11.2, 1.0, 1.0.1 Reporter: Nuutti Kotivuori Priority: Trivial Attachments: couch_log.erl.diff The _log method for couchdb servers has query arguments bytes and offset, but they seem to behave really weirdly. If offset = bytes, an eof error is returned - otherwise offset is just substracted from bytes and the request behaves identically otherwise. A simple fix to the expected behaviour is included - offset counts backwards from bytes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-887) _log handler has very odd semantics for bytes/offset, probably a bug
[ https://issues.apache.org/jira/browse/COUCHDB-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12909083#action_12909083 ] Randall Leeds commented on COUCHDB-887: --- Your patch reverses the semantics of offset, but it's not clear that one is more right. If your intuition about offet being a negative from the end is right, then I'd expect: Start = lists:max([LogFileSize - Offset]) where Bytes determines how many bytes to read, but has no effect on where reading begins. However, in either case I think you're right that Bytes should be passed to file:pread/3 instead of LogFileSize. _log handler has very odd semantics for bytes/offset, probably a bug Key: COUCHDB-887 URL: https://issues.apache.org/jira/browse/COUCHDB-887 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 0.11, 0.11.1, 0.11.2, 1.0, 1.0.1 Reporter: Nuutti Kotivuori Priority: Trivial Attachments: couch_log.erl.diff The _log method for couchdb servers has query arguments bytes and offset, but they seem to behave really weirdly. If offset = bytes, an eof error is returned - otherwise offset is just substracted from bytes and the request behaves identically otherwise. A simple fix to the expected behaviour is included - offset counts backwards from bytes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Addition of modify-on-document-write hooks
Hi James. I think the thing to do is require that a document has a user field, and that the value of that field matches the userCtx in the validate_doc_update function. This then pushes the issue client side, and makes the servers life easier. It could also be added by the front end apache in the case of our deployment, I think. I can see this sort of trigger thing being a good way of giving people a loaded gun aimed at their foot, they certainly are in Oracle if you're not careful. Cheers Simon On 9 Sep 2010, at 05:19, James Jackson wrote: Hi all, Moving this from the users forum, as it appears what I'm after isn't currently available. For the security model I with to implement in a production CouchDB cluster, I would like to be able to force a field to be written to all docs based on the user context. The _update functionality is not what I am after as it requires the user to actually call it when writing a document (means security could be got-around by not calling this, and setting the required field in the passed document to something arbitrary, which would then not get caught by a validation function), and can't modify a document which is passed to it (as far as I can tell it can only modify existing documents, or create new ones). I see this ticket: https://issues.apache.org/jira/browse/COUCHDB-441 which talks about the functionality I am after, but appears to have morphed into what is now there. I am willing to implement such functionality, if it already doesn't exist, but wonder if this would be welcome in the trunk, or if there are killer pitfalls which stop this being possible. I note that in the discussion on that ticket there is talk of how to deal with multiple such modify-on-write functions, perhaps this is one area that needs discussion? In any case, I'll probably implement this for our CouchDB installation, but it would be good to make it generic and globally useful such that I can contribute it back. I know of a number of people who would like this functionality... Regards, James.
Re: Addition of modify-on-document-write hooks
On Sep 13, 2010, at 6:23 PM, Simon Metson wrote: Hi James. I think the thing to do is require that a document has a user field, and that the value of that field matches the userCtx in the validate_doc_update function. This then pushes the issue client side, and makes the servers life easier. It could also be added by the front end apache in the case of our deployment, I think. I can see this sort of trigger thing being a good way of giving people a loaded gun aimed at their foot, they certainly are in Oracle if you're not careful. Cheers Simon The big issue is that any code which runs on normal document updates, will also run during replication, as replication is just a normal client. So this means that adding a field will happen not just on the original client PUT but also when replication happens. This is why _update is a separate handler. Chris On 9 Sep 2010, at 05:19, James Jackson wrote: Hi all, Moving this from the users forum, as it appears what I'm after isn't currently available. For the security model I with to implement in a production CouchDB cluster, I would like to be able to force a field to be written to all docs based on the user context. The _update functionality is not what I am after as it requires the user to actually call it when writing a document (means security could be got-around by not calling this, and setting the required field in the passed document to something arbitrary, which would then not get caught by a validation function), and can't modify a document which is passed to it (as far as I can tell it can only modify existing documents, or create new ones). I see this ticket: https://issues.apache.org/jira/browse/COUCHDB-441 which talks about the functionality I am after, but appears to have morphed into what is now there. I am willing to implement such functionality, if it already doesn't exist, but wonder if this would be welcome in the trunk, or if there are killer pitfalls which stop this being possible. I note that in the discussion on that ticket there is talk of how to deal with multiple such modify-on-write functions, perhaps this is one area that needs discussion? In any case, I'll probably implement this for our CouchDB installation, but it would be good to make it generic and globally useful such that I can contribute it back. I know of a number of people who would like this functionality... Regards, James.
[jira] Commented: (COUCHDB-887) _log handler has very odd semantics for bytes/offset, probably a bug
[ https://issues.apache.org/jira/browse/COUCHDB-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12909103#action_12909103 ] Nuutti Kotivuori commented on COUCHDB-887: -- The *current* semantics of offset are such, that there is really no point in specifying it, ever, as one can just supply a smaller bytes value to get the same effect. If offset is simply substracted from LogFileSize, then offset *must* atleast default to bytes, or otherwise the result is always eof. _log handler has very odd semantics for bytes/offset, probably a bug Key: COUCHDB-887 URL: https://issues.apache.org/jira/browse/COUCHDB-887 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 0.11, 0.11.1, 0.11.2, 1.0, 1.0.1 Reporter: Nuutti Kotivuori Priority: Trivial Attachments: couch_log.erl.diff The _log method for couchdb servers has query arguments bytes and offset, but they seem to behave really weirdly. If offset = bytes, an eof error is returned - otherwise offset is just substracted from bytes and the request behaves identically otherwise. A simple fix to the expected behaviour is included - offset counts backwards from bytes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
