On 6/16/2017 3:08 PM, Ken Gaillot wrote:
On 06/16/2017 01:18 PM, Dan Ragle wrote:
On 6/12/2017 10:30 AM, Ken Gaillot wrote:
On 06/12/2017 09:23 AM, Klaus Wenninger wrote:
On 06/12/2017 04:02 PM, Ken Gaillot wrote:
On 06/10/2017 10:53 AM, Dan Ragle wrote:
So I guess my bottom line question is: How does one tell Pacemaker
that
the individual legs of globally unique clones should *always* be
spread
across the available nodes whenever possible, regardless of the number
of processes on any one of the nodes? For kicks I did try:
pcs constraint location ClusterIP:0 prefers node1-pcs=INFINITY
but it responded with an error about an invalid character (:).
There isn't a way currently. It will try to do that when initially
placing them, but once they've moved together, there's no simple way to
tell them to move. I suppose a workaround might be to create a dummy
resource that you constrain to that node so it looks like the other
node
is less busy.
Another ugly dummy resource idea - maybe less fragile -
and not tried out:
One could have 2 dummy resources that would rather like
to live on different nodes - no issue with primitives - and
do depend collocated on ClusterIP.
Wouldn't that pull them apart once possible?
Sounds like a good idea
Hmmmm... still no luck with this.
Based on your suggestion, I thought this would work (leaving out all the
status displays this time):
# pcs resource create Test1 systemd:test1
# pcs resource create Test2 systemd:test2
# pcs constraint location Test1 prefers node1-pcs=INFINITY
# pcs constraint location Test2 prefers node1-pcs=INFINITY
# pcs resource create Test3 systemd:test3
# pcs resource create Test4 systemd:test4
# pcs constraint location Test3 prefers node1-pcs=INFINITY
# pcs constraint location Test4 prefers node2-pcs=INFINITY
# pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=162.220.75.138
nic=bond0 cidr_netmask=24
# pcs resource meta ClusterIP resource-stickiness=0
# pcs resource clone ClusterIP clone-max=2 clone-node-max=2
globally-unique=true
# pcs constraint colocation add ClusterIP-clone with Test3 INFINITY
# pcs constraint colocation add ClusterIP-clone with Test4 INFINITY
But that simply refuses to run ClusterIP at all ("Resource ClusterIP:0/1
cannot run anywhere"). And if I change the last two colocation
constraints to a numeric then it runs, but with the same problem I had
before (both ClusterIP instances on one node).
I also tried it reversing the colocation definition (add Test3 with
ClusterIP-clone) and trying differing combinations of scores between the
location and colocation constraints, still with no luck.
Thanks,
Dan
Ah of course, the colocation with both means they all have to run on the
same node, which is impossible.
FYI you can create dummy resources with ocf:pacemaker:Dummy so you don't
have to write your own agents.
Good to know, thanks.
OK, this is getting even hackier, but I'm thinking you can use
utilization for this:
http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Pacemaker_Explained/index.html#idm139683960632560
* Create two dummy resources, each with a -INFINITY location preference
for one of the nodes, so each is allowed to run on only one node.
* Set the priority meta-attribute to a positive number on all your real
resources, and leave the dummies at 0 (so if the cluster can't run all
of them, it will stop the dummies first).
* Set placement-strategy=utilization.
* Define a utilization attribute, with values for each node and resource
like this:
** Set a utilization of 1 on all resources except the dummies and the
clone, so that their total utilization is N.
** Set a utilization of 100 on the dummies and the clone.
** Set a utilization capacity of 200 + N on each node.
(I'm assuming you never expect to have more than 99 other resources. If
that's not the case, just raise the 100 usage accordingly.)
With those values, if only one node is up, that node can host all the
real resources (including both clone instances), with the dummies
stopped. If both nodes are up, the only way the cluster can run all
resources (including the clone instances and dummies) is to spread the
clone instances out.
Again, it's hacky, and I haven't tested it, but I think it would work.
Interesting. That does seem to work, at least in my reduction; I've not
yet tried it in my actual real-world setup yet. A few notes, though:
1. I had to set placement-strategy=balanced. When set to utilization the
IP clones still would not split following a standby/unstandby of one of
the nodes.
2. I still had to remember to have resource-stickiness=0 on the
ClusterIP primitives. Without it, after standby/unstandby the clones
still both preferred to stay where they were, with one of the dummies
running on the other node and the second dummy stopped.
3. Rather than set the priority on the "real" resources to 1, I set the
priority on the dummy resources to -1 (so that, going forward, it's one
less thing I'd have to remember to do when adding new resources).
*Seems* to work just as well.
4. If I'm following the logic correctly, I *think* I could set the node
utilization capacity setting to anything LESS THAN the (total clones (2)
+ 1 dummy * 100) and greater than or equal to the number of clones * 100
plus the number of real resources. I.E., for this example anything less
than 300, but greater than or equal to 204 (since I have four Real
resources in my final example). Again, this leaves a little room for the
addition of new resources without having to remember to up the capacity
for the nodes.
All that said, and for anyone interested, here's the recipe I tried that
appears to work well. After setting it up this way, I was able to
standby/unstandby each of the nodes in turn with the clones consistently
re-splitting after each unstandby (and the RealN resources all remaining
on node1 if it was available and node otherwise):
# pcs property set placement-strategy=balanced
# pcs node utilization node1-pcs weight=250
# pcs node utilization node2-pcs weight=250
# pcs resource create Real1 ocf:pacemaker:Dummy
# pcs resource create Real2 ocf:pacemaker:Dummy
# pcs resource create Real3 ocf:pacemaker:Dummy
# pcs resource create Real4 ocf:pacemaker:Dummy
# pcs resource utilization Real1 weight=1
# pcs resource utilization Real2 weight=1
# pcs resource utilization Real3 weight=1
# pcs resource utilization Real4 weight=1
# pcs constraint location Real1 prefers node1-pcs=INFINITY
# pcs constraint location Real2 prefers node1-pcs=INFINITY
# pcs constraint location Real3 prefers node1-pcs=INFINITY
# pcs constraint location Real4 prefers node1-pcs=INFINITY
# pcs resource create Dummy1 ocf:pacemaker:Dummy meta priority=-1
# pcs resource create Dummy2 ocf:pacemaker:Dummy meta priority=-1
# pcs constraint location Dummy1 prefers node2-pcs=-INFINITY
# pcs constraint location Dummy2 prefers node1-pcs=-INFINITY
# pcs resource utilization Dummy1 weight=100
# pcs resource utilization Dummy2 weight=100
# pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=1.2.3.4
nic=bond0 cidr_netmask=24 meta resource-stickiness=0
# pcs resource utilization ClusterIP weight=100
# pcs resource clone ClusterIP clone-max=2 clone-node-max=2
globally-unique=true
Dan
_______________________________________________
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
_______________________________________________
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
