[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530783#comment-13530783 ] Patrick Hunt commented on ZOOKEEPER-1568: - Jimmy this ready for review/commit? If so please submit. Thanks. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: zk-1568_v1.patch, zk-1568_v2.patch Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505588#comment-13505588 ] Jimmy Xiang commented on ZOOKEEPER-1568: I am hacking around this week. I will find out how much performance I can gain this way, if any. Yes, it is mainly for performance and convenience. As to fewer zk events, it may be just for our use case, assuming our node children changed handler doesn't have a chance to reset the watch soon enough. So if we create lots of children for one parent, we may get less node children changed events, theoretically. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Attachments: zk-1568_v1.patch Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13484256#comment-13484256 ] Jimmy Xiang commented on ZOOKEEPER-1568: For our use case, there is no dependency issue. Batch is what we want. Another benefit is that we will get less ZK events. This may not be obvious. But we will do get less ZK events. For example, for a bunch of create operations, if we do it one by one, once we get a nodeChildrenChanged event, we will watch it again, so we will get another one for the next create operation. If they are batched, after we get the first nodeChildrenChanged event, when we watch it again, most likely, other nodes are already created, so we will get less events, which is good. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483136#comment-13483136 ] Flavio Junqueira commented on ZOOKEEPER-1568: - Hi Jimmy, I'm trying to understand why submitting operations asynchronously is not sufficient for your case. Why do you need to use multi in this case? multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483311#comment-13483311 ] Jimmy Xiang commented on ZOOKEEPER-1568: Hi Flavio, for our use case, we need to create/setData hundreds/thousands of znodes. By submitting operations asynchronously, we need to do it one by one. If we can do it in batches, we can save lots of network trips. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483347#comment-13483347 ] Flavio Junqueira commented on ZOOKEEPER-1568: - In my view, the asynchronous API has been designed to address exactly use cases like yours. I don't think you should be suffering any severe penalty by using the asynchronous API. Have you actually tried it and had any issue with it? multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483590#comment-13483590 ] Marshall McMullen commented on ZOOKEEPER-1568: -- I actually think there is a valid use case for this. Mostly for performance reasons. Because a multi is one transaction, it causes less permuation on the distributed and replicated state of zookeeper than multiple individual operations not in a multi. With a Multi: - You only pay the cost of the RPC overhead once rather than on each individual operation - You get one flush of the leader channel rather than multiple ones for each write operation - A multi will case one new snapshot/log to be generated rather than multiple ones for each operation There are other reasons that make this a good reason too that are not performance based. e.g., if it makes the programmer's job easier to use a multi with these semantics, then that's a win. In other distributed databases I've worked on, we used different terminology to disinguish between a multi op that all succeed/fail vs one that does not. We used the term Batch to imply we were batching up operations but there was no guarantee they'd all succeed/fail. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483618#comment-13483618 ] Ted Yu commented on ZOOKEEPER-1568: --- bq. A multi will case one new snapshot/log to be generated I guess you meant 'cause' above. bq. but there was no guarantee they'd all succeed/fail. I think we need to formalize how success / failure status for individual operations in this new multi API should be delivered back to client. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483626#comment-13483626 ] Marshall McMullen commented on ZOOKEEPER-1568: -- Yes, I meant 'cause' :). The existing multi code fills in a list of results for each op. Right now, it aborts on the first op that fails and rolls back the data tree to what it was before it started. And it explicitly marks all ops after that in the results list with a runtime exception. So the mechanism is already there to communicate the errors back to the client. So I suppose the Multi code would need to take a bool to indicate if it was all or nothing or not. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1568) multi should have a non-transaction version
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483635#comment-13483635 ] Ted Yu commented on ZOOKEEPER-1568: --- bq. it aborts on the first op that fails and rolls back Should we allow operations after the failed operation to continue ? The rationale is that the operations in the batch may not have dependencies among them. multi should have a non-transaction version --- Key: ZOOKEEPER-1568 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1568 Project: ZooKeeper Issue Type: Improvement Reporter: Jimmy Xiang Currently multi is transactional, i.e. all or none. However, sometimes, we don't want that. We want all operations to be executed. Even some operation(s) fails, it is ok. We just need to know the result of each operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
