Furtunately the DRBD guys were able to help me with this. I had a
wrong timeout set in the drbd-peer-outdater.
Thanks for the efforts you put in trying to help me.
Christoph
2008/9/18 Andrew Beekhof <[EMAIL PROTECTED]>:
> On Thu, Sep 18, 2008 at 10:34, Christoph Eßer <[EMAIL PROTECTED]> wrote:
>> 2008/9/17 Andrew Beekhof <[EMAIL PROTECTED]>:
>>> On Wed, Sep 17, 2008 at 16:06, Christoph Eßer <[EMAIL PROTECTED]> wrote:
>>>> I don't see why it should not be. cat /proc/drbd indicates the DRBD
>>>> resource is working correctly on both nodes.
>>>
>>> Call me crazy, but I thought it wise to ask since drbd is spewing out logs
>>> like:
>>> Sep 16 13:50:16 viktor-01 drbd[4405]: [4424]: DEBUG: r0: Command
>>> output: No response from the DRBD driver! Is the module loaded?
>>>
>>> I'd honestly start with the drbd guys first... I can't imagine what we
>> Ok I went to the DRDB guys and asked for help. But I still have the
>> impression that this problem is related to Linux-HA.
>
> It may well be, but what possible command could we give it to make the
> drbd module, on the local machine, stop responding to the drbd admin
> client? That seems to me like something that shouldn't be possible
> (ie. a drbd bug) even if it somehow is.
>
>>
>> I added a full log of the reboot process on my slave machine. Maybe
>> this helps finding the problem.
>>
>> Thanks
>>
>> Christoph
>>
>>> could have done to cause drbd to behave in this way.
>>> But then I'm not a drbd user either...
>>>
>>>>
>>>> Andrew Beekhof wrote:
>>>>> Stupid question, is the driver in fact loaded?
>>>>>
>>>>> On Tue, Sep 16, 2008 at 16:17, Christoph Eßer <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> I am using Heartbeat 2.99 and Pacemaker 0.6.6 on Debian Lenny.
>>>>>>
>>>>>> I think my problem is anyhow related to DRBD but as I only encounter it
>>>>>> using heartbeat and pacemaker I'll post it here.
>>>>>>
>>>>>> Anything works as expected if I just force the system to failover by
>>>>>> unplugging the power supply of any of my two nodes. Shutting down the
>>>>>> DRBD
>>>>>> slave via "init 0" or "init 6" works fine as well.
>>>>>>
>>>>>> But whenever I restart my DRBD master via "init 6" the whole cluster
>>>>>> crashes
>>>>>> and fails to reassign the resources, even after the node started up
>>>>>> again.
>>>>>> Anyway, I found some strange DRBD related messages in /var/log/syslog on
>>>>>> the
>>>>>> second node while restarting the master node:
>>>>>>
>>>>>> Sep 16 13:48:44 viktor-01 drbd[4123]: [4135]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:48:44 viktor-01 drbd[4123]: [4151]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:49:14 viktor-01 drbd[4164]: [4176]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:49:14 viktor-01 drbd[4164]: [4192]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:49:36 viktor-01 drbd[4226]: [4239]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:49:36 viktor-01 drbd[4226]: [4255]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:49:41 viktor-01 drbd[4283]: [4296]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:49:41 viktor-01 drbd[4283]: [4313]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:49:54 viktor-01 drbd[4348]: [4360]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:49:54 viktor-01 drbd[4348]: [4376]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:49:54 viktor-01 drbd[4348]: [4386]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf primary r0
>>>>>> Sep 16 13:49:54 viktor-01 kernel: [ 495.079064] drbd0: helper command:
>>>>>> /sbin/drbdadm outdate-peer
>>>>>> Sep 16 13:50:15 viktor-01 drbd[4405]: [4416]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf secondary r0
>>>>>> Sep 16 13:50:16 viktor-01 drbd[4405]: [4424]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 secondary' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:16 viktor-01 lrmd: [2279]: info: RA output:
>>>>>> (drbd:1:demote:stdout) No response from the DRBD driver! Is the module
>>>>>> loaded? Command 'drbdsetup /dev/drbd0 secondary' terminated with exit
>>>>>> code
>>>>>> 20 drbdadm aborting
>>>>>> Sep 16 13:50:16 viktor-01 drbd[4426]: [4439]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:17 viktor-01 drbd[4426]: [4447]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:17 viktor-01 drbd[4426]: [4455]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:17 viktor-01 drbd[4426]: [4463]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:17 viktor-01 drbd[4426]: [4464]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:19 viktor-01 drbd[4482]: [4494]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:19 viktor-01 drbd[4482]: [4502]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:19 viktor-01 drbd[4482]: [4510]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:20 viktor-01 drbd[4482]: [4518]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:20 viktor-01 drbd[4482]: [4519]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:21 viktor-01 drbd[4526]: [4538]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:21 viktor-01 drbd[4526]: [4546]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:21 viktor-01 drbd[4526]: [4554]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:22 viktor-01 drbd[4526]: [4562]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:22 viktor-01 drbd[4526]: [4563]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:22 viktor-01 drbd[4526]: [4564]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf up r0
>>>>>> Sep 16 13:50:22 viktor-01 drbd[4526]: [4572]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 disk /dev/cciss/c0d0p4 /dev/cciss/c0d0p4 internal
>>>>>> --set-defaults
>>>>>> --create-device --fencing=resource-only' terminated with exit code 20
>>>>>> Sep 16 13:50:22 viktor-01 lrmd: [2279]: info: RA output:
>>>>>> (drbd:0:start:stdout) No response from the DRBD driver! Is the module
>>>>>> loaded? Command 'drbdsetup /dev/drbd0 disk /dev/cciss/c0d0p4
>>>>>> /dev/cciss/c0d0p4 internal --set-defaults --create-device
>>>>>> --fencing=resource-only' terminated with exit code 20
>>>>>> Sep 16 13:50:22 viktor-01 drbd[4526]: [4574]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:23 viktor-01 drbd[4526]: [4582]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:23 viktor-01 drbd[4526]: [4590]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:23 viktor-01 drbd[4526]: [4598]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:23 viktor-01 drbd[4526]: [4599]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:25 viktor-01 drbd[4613]: [4625]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:25 viktor-01 drbd[4613]: [4633]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:25 viktor-01 drbd[4613]: [4641]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:26 viktor-01 drbd[4613]: [4649]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:26 viktor-01 drbd[4613]: [4650]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:26 viktor-01 drbd[4657]: [4669]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf state r0
>>>>>> Sep 16 13:50:27 viktor-01 drbd[4657]: [4677]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:27 viktor-01 drbd[4657]: [4685]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf cstate r0
>>>>>> Sep 16 13:50:27 viktor-01 drbd[4657]: [4693]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
>>>>>> Sep 16 13:50:27 viktor-01 drbd[4657]: [4694]: DEBUG: r0 status: No
>>>>>> response
>>>>>> from the DRBD driver! Is the module loaded? Command 'drbdsetup /dev/drbd0
>>>>>> state' terminated with exit code 20 drbdadm aborting Not configured Not
>>>>>> configured Unconfigured
>>>>>> Sep 16 13:50:27 viktor-01 drbd[4657]: [4695]: DEBUG: r0: Calling drbdadm
>>>>>> -c
>>>>>> /etc/drbd.conf up r0
>>>>>> Sep 16 13:50:28 viktor-01 drbd[4657]: [4703]: DEBUG: r0: Command output:
>>>>>> No
>>>>>> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
>>>>>> /dev/drbd0 disk /dev/cciss/c0d0p4 /dev/cciss/c0d0p4 internal
>>>>>> --set-defaults
>>>>>> --create-device --fencing=resource-only' terminated with exit code 20
>>>>>>
>>>>>>
>>>>>> I don't see any reason for these errors. I would appreciate anyone
>>>>>> helping
>>>>>> me to find a starting point for a solution.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Christoph
>>>>>>
>>>>>> resource r0 {
>>>>>> protocol C;
>>>>>> # incon-degr-cmd "killall heartbeat"; #"halt -f"; # killall heartbeat
>>>>>> would
>>>>>> be a good alternative :->
>>>>>>
>>>>>> syncer {
>>>>>> rate 40M; # Note: 'M' is MegaBytes/s, not MegaBits/s; recommended is
>>>>>> 30 %
>>>>>> of maximum throughput (considering both network and hard disk access)
>>>>>> }
>>>>>>
>>>>>>
>>>>>> on viktor-01 {
>>>>>> disk /dev/cciss/c0d0p4;
>>>>>> device /dev/drbd0;
>>>>>> meta-disk internal;
>>>>>> address 172.16.1.1:7789;
>>>>>> }
>>>>>> on viktor-02 {
>>>>>> disk /dev/cciss/c0d0p4;
>>>>>> device /dev/drbd0;
>>>>>> meta-disk internal;
>>>>>> address 172.16.1.2:7789;
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> ucast eth0 172.16.0.1
>>>>>> ucast eth0 172.16.0.2
>>>>>> ucast eth1 172.16.1.1
>>>>>> ucast eth1 172.16.1.2
>>>>>>
>>>>>> node viktor-01 viktor-02
>>>>>> crm on
>>>>>> respawn hacluster /usr/lib/heartbeat/dopd
>>>>>> apiauth dopd gid=haclient uid=hacluster
>>>>>>
>>>>>> _______________________________________________
>>>>>> Linux-HA mailing list
>>>>>> [email protected]
>>>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>>>> See also: http://linux-ha.org/ReportingProblems
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Linux-HA mailing list
>>>>> [email protected]
>>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>>> See also: http://linux-ha.org/ReportingProblems
>>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-HA mailing list
>>>> [email protected]
>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>> See also: http://linux-ha.org/ReportingProblems
>>>>
>>> _______________________________________________
>>> Linux-HA mailing list
>>> [email protected]
>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>> See also: http://linux-ha.org/ReportingProblems
>>>
>> _______________________________________________
>> Linux-HA mailing list
>> [email protected]
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>>
> _______________________________________________
> Linux-HA mailing list
> [email protected]
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
