Re: [Discuss] Management cluster / Zookeeper holding locks

Now, yes! Thanks for the clarification. On Mon, Dec 18, 2017 at 11:16 AM, Marc-Aurèle Brothier wrote: > Sorry about the confusion. It's not going to replace the DB transactions in > the DAO way. Today we can say that there are 2 types of locks in CS, either > a pure

Re: [Discuss] Management cluster / Zookeeper holding locks

Sorry about the confusion. It's not going to replace the DB transactions in the DAO way. Today we can say that there are 2 types of locks in CS, either a pure transaction one, with the select for update which locks a row for any operation by other threads, or a more programmatic one with the

Re: [Discuss] Management cluster / Zookeeper holding locks

So, we would need to change every piece of code that opens and uses connections and transactions to change to ZK model? I mean, to direct the flow to ZK. On Mon, Dec 18, 2017 at 8:55 AM, Marc-Aurèle Brothier wrote: > I understand your point, but there isn't any "transaction"

Re: [Discuss] Management cluster / Zookeeper holding locks

I understand your point, but there isn't any "transaction" in ZK. The transaction and commit stuff are really for DB and not part of ZK. All entries (if you start writing data in some nodes) are versioned. For example you could enforce that to overwrite a node value you must submit the node data

Re: [Discuss] Management cluster / Zookeeper holding locks

I did not check the link before. Sorry about that. Reading some of the pages there, I see curator more like a client library such as MySQL JDBC client. When I mentioned framework, I was looking for something like Spring-data. So, we could simply rely on the framework to manage connections and

Re: [Discuss] Management cluster / Zookeeper holding locks

@rafael, yes there is a framework (curator), it's the link I posted in my first message: https://curator.apache.org/curator-recipes/shared-lock.html This framework helps handling all the complexity of ZK. The ZK client stays connected all the time (as the DB connection pool), and only one

Re: [Discuss] Management cluster / Zookeeper holding locks

Do we have framework to do this kind of looking in ZK? I mean, you said " create a new InterProcessSemaphoreMutex which handles the locking mechanism.". This feels that we would have to continue opening and closing this transaction manually, which is what causes a lot of our headaches with

Re: [Discuss] Management cluster / Zookeeper holding locks

We added ZK lock for fix this issue but we will remove all current locks in ZK in favor of ZK one. The ZK lock is already encapsulated in a project with an interface, but more work should be done to have a proper interface for locks which could be implemented with the "tool" you want, either a DB

Re: [Discuss] Management cluster / Zookeeper holding locks

so, how does that work? I mean, instead of opening a transaction with the database and executing locks, what do we need to do in the code? On Mon, Dec 18, 2017 at 7:24 AM, Ivan Kudryavtsev wrote: > Rafael, > > - It's easy to configure and run ZK either in single node

Re: [Discuss] Management cluster / Zookeeper holding locks

Rafael, - It's easy to configure and run ZK either in single node or cluster - zookeeper should replace mysql locking mechanism used inside ACS code (places where ACS locks tables or rows). I don't think from the other size, that moving from MySQL locks to ZK locks is easy and light and (even

Re: [Discuss] Management cluster / Zookeeper holding locks

How hard is it to configure Zookeeper and get everything up and running? BTW: what zookeeper would be managing? CloudStack management servers or MySQL nodes? On Mon, Dec 18, 2017 at 7:13 AM, Ivan Kudryavtsev wrote: > Hello, Marc-Aurele, I strongly believe that all

Re: [Discuss] Management cluster / Zookeeper holding locks

Hello, Marc-Aurele, I strongly believe that all mysql locks should be removed in favour of truly DLM solution like Zookeeper. The performance of 3node ZK ensemble should be enough to hold up to 1000-2000 locks per second and it helps to move to truly clustered MySQL like galera without single

Re: [Discuss] Management cluster / Zookeeper holding locks

Are you proposing to add zookeeper as an optional requirement, Marc-Aurèle? or just curator? and what is the decision mech of including it or not? On Mon, Dec 18, 2017 at 9:33 AM, Marc-Aurèle Brothier wrote: > Hi everyone, > > I was wondering how many of you are running

[Discuss] Management cluster / Zookeeper holding locks

Hi everyone, I was wondering how many of you are running CloudStack with a cluster of management servers. I would think most of you, but it would be nice to hear everyone voices. And do you get hosts going over their capacity limits? We discovered that during the VM allocation, if you get a lot